mongo-express: import does not work on json arrays files.

I still can not use the import function. Please state that the import feature is not supported or remove the import button.

@pista37 we also accept PRs if that’s really important to you

Seems like there are a lot of seemingly hacky solutions to this problem. Why can mongo-express export to a format and not import the same format? Seems extremely convoluted.

Steps to manually avoid this on docker

1. Export JSON from mongoexpress
2. Copy exported json into mongodb container docker cp <src-import-file> <mongodb container name>:<import src destination>
3. Authenticate into mongodb container and import using mongoimport docker-compose exec <mongodb container name> mongoimport --jsonArray -d <database> -c <collection> --authenticationDatabase admin --username=<user> --password=<pass> <import src destination>

If there are any maintainers looking through here PLEASE address this bug. I straight up would’ve gone with another mongo gui had I known the json import/export didn’t work. Very misleading to even have that button if there’s no way to import without jumping through a ton of hoops.

Since I’m using docker I’ve tried to overwrite the collection.js file using all of the pieces above and so far nothing has worked. The above pieces seem to address the $oid key but not the $date key. If I’m feeling saucy I might get this figured out and submit a PR but it’s honestly shocking that this has existed for so long and nobody has taken the reigns. Especially with all of the people here fixing it on their local branch.

I also cannot get import to work. Seems like a pretty broken feature at the moment.

@mike442144 but it doesn’t work with mongo-express’ own export function, right? I tried to export one collection using the export button, and importing it again using the import button

If I export with the “standard” option then on import it fails saying Error: key $oid must not start with '$'. If I export as a “jsonArray” then it fails saying operation passed in cannot be an Array.

mongoimport --jsonArray is working, but I would prefer to use Mongo Express.

I am using json file which generated by export function of mongo-express but i am still receiving same error.

UPDATE I try to this command for import, its worked for me. docker-compose exec mongo mongoimport -d *databasename* -c *collection_name* *json_path*

I updated my docker image to 51.2 and I still can’t get the imports and exports to work…

https://github.com/mongo-express/mongo-express/blob/797e68de25e3566f658b3691ddbb2aac20e393c0/lib/routes/collection.js#L576-L577

I modified collection.js in my working clone and it works for me…

var fileContent = req.files[key].data.toString('utf8');
if (fileContent.slice(0,1) === '[' && fileContent.slice(-1) === ']') fileContent = fileContent.slice(1,-1);
const str = `[${fileContent.replace(/}[\r\n]{1,2}(\s)?\{/gm, '},{').replace(/\{\s*"\$oid"\s*:\s*"(.+?)"\s*}/gm, '"$1"')}]`;

No clue about the tool you use, but perhaps try this (wrap your object into array):

[{
	"_id": "111112",
	"name": "art",
	"orders": []
}]

For all of you that are struggling with export/import, I think the simplest way to fix this is to just open vim. Then start a macro, do the edits whatever it might be. Then repeat it for the whole document. As an example, let’s assume the below export content. It contains the object ID, and a date value for key lastModified. If you have a pretty format you will first want to minify the output with a simple tool like: https://www.cleancss.com/json-minify/ otherwise the macro that I use will not work.

[{"_id":{"$oid":"632b3b44d88c0a88eb307481"},"lastModified":{"$date":"2022-10-06T03:58:14.291Z"}},{"_id":{"$oid":"632b3b44d88c0a88eb307482"},"lastModified":{"$date":"2022-10-07T03:58:14.291Z"}},{"_id":{"$oid":"632b3b44d88c0a88eb307483"},"lastModified":{"$date":"2022-10-08T03:58:14.291Z"}}]

To make this import file work, you will need to replace the $oid with ObjectId() and also replace the $date with ISODate(). If your export file contains other keys you will have to do the same thing for them. I don’t know about other formats that require transformation. But for this specific example I use the below macro:

First open the file in vim editor

qq       (to start macro under letter q)

gg       (go to document start)
0        (go to start of line)
/\$oid   (start a forward search for $oid)
f:       (search for next colon)
,        (reverse search for previous colon)
l        (go one position/char to the right)
v        (visual mode)
f:       (search for next colon)
s        (start substitution)
ObjectId(
ESC
f}       (find closing curly brace)
s        (start substitution)
)

gg       (go to document start)
0        (go to start of line)
/\$date  (start a forward search for $date)
f:       (search for next colon)
,        (reverse search for previous colon)
l        (go one position/char to the right)
v        (visual mode)
f:       (search for next colon)
s        (start substitution)
ISODate(
ESC
f}       (find closing curly brace)
s        (start substitution)
)

q        (stop the recording)

1000@q   (depending on how large your export is, you can define to execute it X times. In this case I execute the macro a thousand times.).

After the macro is done the new file content will be:

[{"_id":ObjectId("632b3b44d88c0a88eb307481"),"lastModified":ISODate("2022-10-06T03:58:14.291Z")},{"_id":ObjectId("632b3b44d88c0a88eb307482"),"lastModified":ISODate("2022-10-07T03:58:14.291Z")},{"_id":ObjectId("632b3b44d88c0a88eb307483"),"lastModified":ISODate("2022-10-08T03:58:14.291Z")}]

And after importing you have the following documents:

{
    _id: ObjectId('632b3b44d88c0a88eb307481'),
    lastModified: ISODate('2022-10-06T03:58:14.291Z')
},
{
    _id: ObjectId('632b3b44d88c0a88eb307482'),
    lastModified: ISODate('2022-10-07T03:58:14.291Z')
},
{
    _id: ObjectId('632b3b44d88c0a88eb307483'),
    lastModified: ISODate('2022-10-08T03:58:14.291Z')
}

I guess if you need to do export/import often, then you might want to write a simple script in whatever language you prefer, and execute the same search/replace. For me above is sufficient for those few occasions.

I’m here 😃. Actually, It is safe to use the format exported from shell command mongoexport. I have noted on the button in a commit. So if you have data from your script or third-party, transform before import. If you have no idea about the format, try typing mongoexport in shell 😃