Support a number as key in object when packing as MAP?

[majianjia]

Hi,

I am developing a protocol in a resource constrain platform and suffer from very limited bandwidth.
We are using map(integer: any) to transfer messages.

However, this lib only packs objects as map() while the "key" in the object is required to be a string.
Using string adds at least additional 2 bytes compared to the integer, while the map(integer: bool or integer) can be only 3 bytes.

I understand this request is not commonly used in the web dev, but it is very useful for us to reduce the bandwidth.

Thanks,

[gfx]

You can already use an Array as the data source of encode(), so once you "translate" your objects into arrays before passing them to encode(), the encoded objects will get smaller (and IIUC that's what ProtocolBuffers does).

[majianjia]

Thanks for the reply, I will try this method.

[benatkin]

I think this is important for interoperability, especially with Python, where integer keys are very common in maps.

To support I would perhaps have three: one that uses plain objects, one that uses Map, and one that calls a function asking whether to use a plain object or a Map when serializing/deserializing.

[3da]

I have the same issue. Would be fine if we could annotate fields in some TS interface and then it translated automatically into an object with such interface.

[yayiyao]

same question. Object will automatically convert keys into string type, and Map objects are not supported. How to solve?

object or Map need to encode:

{ 1: 'hello', 2: 'world' }

[Aetherus]

This is extremely important when encoding sparse arrays with very large indices, e.g. {45319: "foo", 97813: "bar"} and {[123, 321]: 0.1, [222, 333]: 0.2}

[Yotsubal]

I use this code to approach this issue, by creating custom encoder

  var Hex = "816501"; // {101: 1}

  var MsgPackEncoder = require("@msgpack/msgpack").Encoder
  var MsgPack_encoderWithNumberMap = (value, options) => {
      const encoder = new MsgPackEncoder(options);
      encoder.encodeMap = (object, depth) => {
        const keys = Object.keys(object);
        if (encoder.sortKeys) {
            keys.sort();
        }
        const size = encoder.ignoreUndefined ? encoder.countWithoutUndefined(object, keys) : keys.length;
        if (size < 16) {
            // fixmap
            encoder.writeU8(0x80 + size);
        }
        else if (size < 0x10000) {
            // map 16
            encoder.writeU8(0xde);
            encoder.writeU16(size);
        }
        else if (size < 0x100000000) {
            // map 32
            encoder.writeU8(0xdf);
            encoder.writeU32(size);
        }
        else {
            throw new Error(`Too large map object: ${size}`);
        }
        for (const key of keys) {
            const value = object[key];
            if (!(encoder.ignoreUndefined && value === undefined)) {
              // Check if Key String is a number
              if(Number(key) == key) {
                encoder.encodeNumber(Number(key));
              } else {
                encoder.encodeString(key);
              }
              encoder.doEncode(value, depth + 1);
            }
        }
      }
      return encoder.encodeSharedRef(value);
  }

  var B1 = msgPack.decode(Buffer.from(Hex, "hex"))
  var B2 = MsgPack_encoderWithNumberMap(B1, {})
  var B3 = msgPack.encode(B1, {})
  
  console.log(Buffer.from(B2).toString("hex") == Hex) // true - 816501 
  console.log(Buffer.from(B3).toString("hex") == Hex) // false - 81a331303101

[Zhang-Junzhi]

I’d like to add that this issue is equally critical for unpacking, not just packing.
When receiving data from other languages (like Python) where integer keys are standard, simply deserializing into a JS Object causes data loss because JS coerces integer keys to strings.
My suggestion:
During unpack, the library should detect maps with integer keys and automatically convert them to a JavaScript Map object. Alternatively, providing an option (e.g., useMapForIntKeys: true) to force this behavior would be a great solution to ensure full interoperability.