projen: Using `--from` w/ file or remote repository format fails when the source descends from PythonProject.

Assuming we have a simple project defined locally as follows (that descends from JSIIProject):

in templates/src/index.ts:

export class MyProject extends python.PythonProject {                                                                                                                                                                                                                                                                                                                
  constructor(options: python.PythonProjectOptions) {                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
    super(options);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
  }                                                                                                                                                                                                                                                                                                                                                                       
}

After building the project and compiling the JSII, when I try to use the template from another project directory, such as new_project:

projen new --from file:../templates

I get the following error:

Error: Cannot find module 'templates/.jsii'

If you try the same using a remote repository, you will get:

Error: Cannot find module 'git/.jsii'

I looked into the code, and it looks like this is erroring at this point:

https://github.com/projen/projen/blob/9f4a6ffa9ab90400fbbecdda7d993290b391ebb2/src/cli/cmds/new.ts#L211-L213

Installing from an NPM or private NPM server still works.

About this issue

Original URL
State: open
Created 2 years ago
Comments: 15 (6 by maintainers)

Most upvoted comments

@orlandronen1 thanks for the tip, renaming the tarball did work

works:

npx projen new zaiPython --packageManager npm --from /home/zmully/gitwork/python-test-2/dist/js/python-test-2.tgz

does not work:

npx projen new zaiPython --packageManager npm --from python-test-2@/home/zmully/gitwork/python-test-2/dist/js/python-test-2.tgz
npx projen new zaiPython --packageManager npm --from "python-test-2@/home/zmully/gitwork/python-test-2/dist/js/python-test-2.tgz"
npx projen new zaiPython --packageManager npm --from "python-test-2@/home/zmully/gitwork/python-test-2/dist/js/python-test-2@0.0.0.jsii.tgz"

And like you said, pulling from a registry solves many of these issues, but it’d be great if that friction could be removed to speed up development.

zmully on Sep 14, 2022

zmully@ubu2 /tmp/tmp.LjeUntIV5c $ npm -v
8.15.0

@mrgrain still not working with the package name addition:

zmully@ubu2 /tmp/tmp.WsRcs7o6OH $ cd $(mktemp -d) && npx projen new zaiPython --packageManager npm --from python-test-2@/home/zmully/gitwork/python-test-2/dist/js/python-test-2@0.0.0.jsii.tgz 
/tmp/tmp.LjeUntIV5c
└── (empty)


added 1 package, and audited 68 packages in 1s

7 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
👾 installing external module python-test-2@/home/zmully/gitwork/python-test-2/dist/js/python-test-2@0.0.0.jsii.tgz...

added 1 package, and audited 69 packages in 385ms

7 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
👾 Project definition file was created at /tmp/tmp.LjeUntIV5c/.projenrc.py
/tmp/tmp.LjeUntIV5c/node_modules/projen/node_modules/semver/classes/comparator.js:38
      throw new TypeError(`Invalid comparator: ${comp}`)
      ^

TypeError: Invalid comparator: /home/zmully/gitwork/python-test-2/dist/js/python-test-2@0.0.0.jsii.tgz
    at Comparator.parse (/tmp/tmp.LjeUntIV5c/node_modules/projen/node_modules/semver/classes/comparator.js:38:13)
    at new Comparator (/tmp/tmp.LjeUntIV5c/node_modules/projen/node_modules/semver/classes/comparator.js:22:10)
    at /tmp/tmp.LjeUntIV5c/node_modules/projen/node_modules/semver/classes/range.js:133:47
    at Array.map (<anonymous>)
    at Range.parseRange (/tmp/tmp.LjeUntIV5c/node_modules/projen/node_modules/semver/classes/range.js:133:35)
    at /tmp/tmp.LjeUntIV5c/node_modules/projen/node_modules/semver/classes/range.js:34:22
    at Array.map (<anonymous>)
    at new Range (/tmp/tmp.LjeUntIV5c/node_modules/projen/node_modules/semver/classes/range.js:34:8)
    at Object.toPythonVersionRange (/tmp/tmp.LjeUntIV5c/node_modules/projen/lib/util/semver.js:51:19)
    at RequirementsFile.formatDependency (/tmp/tmp.LjeUntIV5c/node_modules/projen/lib/python/requirements-file.js:45:43)

zmully on Sep 6, 2022

Ah yes, so projen parses the this “spec” differently than npm. While npm is clever enough to figure out that this is a path, we don’t do this check in projen:

https://github.dev/projen/projen/blob/274682b35bd13ccacfb8444270e85c891838ea19/src/cli/cmds/new.ts#L283-L285 The whole path is just added as dependency, which later on will be split at the @ and that’s how it ends up in package.json.

I think instead of blindly adding the spec string as dev dependency, we need to check against the moduleName that’s returned from the install helper. Or even discard spec and use whatever name/version npm comes back with.

In the meantime, I also noticed this comment: https://github.dev/projen/projen/blob/274682b35bd13ccacfb8444270e85c891838ea19/src/cli/util.ts#L6-L11

And for me projen new --from @mb/cdk-project-backbone@/Users/tymoteuszgach/git/tyga/cdk-project-backbone/dist/js/cdk-project-backbone@0.0.0.jsii.tgz works!

mrgrain on Aug 29, 2022