matrix-js-sdk: UploadContent m.image uploads won't embed consistently across platforms (Element Web)

When using some basic script to test the upload functionality of this sdk, I am met with a weird problem. Image uploads will have varying behavior based upon the matrix client in use.

For example, Element Web will display my sdk image uploads as a blank line. I can download the image and it will display fine, but no embedding is done with Element Web. For Element Mobile, embeds work fine. I’ve tested this functionality across multiple users, with multiple web and mobile clients, across multiple matrix servers and rooms.

I’m using a very basic test program to demonstrate the concept:

import sdk from 'matrix-js-sdk';
import path from 'path';
import fs from 'fs';
import fetch from 'node-fetch';

const filename = "/path/to/file.jpg";
const filepath = path.basename(filename);

var stream = fs.createReadStream(filename);
console.log(filename);

const client = sdk.createClient({
    baseUrl: "URL",
    accessToken: "accesstoken",
    userId: "@user:URL"
});

client.startClient();

client.once('sync', function(state, prevState, res) {
    console.log(state); 
});

var roomid = "!roomID:URL"


client.uploadContent(stream, {
}).then(function(url) {
	var content = {
		msgtype: "m.image",
		body: filename,
		url: url.content_uri
};
client.sendMessage(roomid, content);

I’ve experimented with adding a delay to wait for a, “PREPARED” state from the room, but that makes no difference. I’ve also tried a few options from uploadContent out of curiosity but they also make no difference.

When inspecting the source in Element Web for the image (the one which is uploaded by this sdk, which will fail to embed on Element Web), I see the folllowing:

{
  "type": "m.room.message",
  "sender": "user:URL",
  "content": {
    "msgtype": "m.image",
    "body": "image.jpg",
    "url": "mxc://URL/SomeString"
  },
  "origin_server_ts": SomeNums,
  "unsigned": {
    "age": SomeOtherNums
  },
  "event_id": "$EventID",
  "room_id": "!RoomID:URL"
}

Comparing to another image which was uploaded without metadata which DOES embed in Element Web (not uploaded with this SDK, different image which has no metadata), we can see the following source in Element:

{
  "content": {
    "body": "image.jpeg",
    "msgtype": "m.image",
    "url": "mxc://URL/SomeString"
  },
  "origin_server_ts": SomeNums,
  "room_id": "!roomID:URL",
  "sender": "user:URL",
  "type": "m.room.message",
  "unsigned": {},
  "event_id": "$eventID",
  "user_id": "user:URL"
}

And, here’s the source for the same image as found in the first source, but uploaded manually by a user (which will embed in Element Web):

{
  "type": "m.room.message",
  "sender": "user:URL",
  "content": {
    "body": "image.jpg",
    "info": {
      "size": 282691,
      "mimetype": "image/jpeg",
      "thumbnail_info": {
        "w": 450,
        "h": 600,
        "mimetype": "image/jpeg",
        "size": 121384
      },
      "w": 1240,
      "h": 1653,
      "xyz.amorgan.blurhash": "Somehash",
      "thumbnail_url": "mxc://URL/SomeNums"
    },
    "msgtype": "m.image",
    "url": "mxc://URL/SomeNums"
  },
  "origin_server_ts": SomeOtherNums,
  "unsigned": {
    "age": Age
  },
  "event_id": "$EventID",
  "room_id": "!RoomID:URL"
}

There’s a fair bit of difference within the, “source info” for these three images. Clearly based on the second source, the metadata (such as height and width) isn’t necessary for the image to embed in Element Web. However, there are other differences present between source 1 and source 2.

Is this an error on my behalf on how I am uploading the image using this SDK, or is this a problem within this sdk? Is there any further information that I could provide to help debug this problem?

Thanks

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 20 (9 by maintainers)

Most upvoted comments

Thanks for the explanations.

Adding a mimetype seems to have fixed it!

client.uploadContent(blob, {
        name: "filename.mp4",
        type: "video/mp4"
}).then(function(url) {
        var content = {
                msgtype: "m.video",
                body: "filename.mp4",
                info : {
                        mimetype: "video/mp4"
                },
                url: url.content_uri
};

I had tried adding a mimetype before, but I didn’t know it was a part of info or if this SDK supported it.

I’ll go ahead and test this for a few days, and I’ll close this issue if everything is working well. Thank you so much for the help, I’m happy that I could find a working solution to this issue!

Thank you. I was tearing my hair out over this one. For the record, here’s the code I had to use to make it work.

	public void sendImage(String roomID, String message, String imgUrl, DataCallback response) throws IOException {
		if (!isLoggedIn)
			return;

		JsonObject data = new JsonObject();
		JsonObject info = new JsonObject();
		data.addProperty("msgtype", "m.image");
		data.addProperty("body", message);
		info.addProperty("mimetype",  "image/png");
		data.add("info", info);
		data.addProperty("url", imgUrl.replaceAll("\"", "").trim());
		
		sendRoomEvent("m.room.message", roomID, data, response);
	}