Cannot find package when using ES Module and Lambda Layer

[paul-uz]

I am testing out using the new ES Module system, along with a Lambda Layer.

Here is my tsconfig file I used to compile my index.js file:

{
  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "suppressImplicitAnyIndexErrors": true,
    "moduleResolution": "node",
    "paths": {
      "*": [
        "../../node_modules/*"
      ]
    },
  },
}

I set the type to module in the lambda functions package.json

{
  "name": "foobar",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Here is my package.json file I use for the Lambda Layer:

{
  "name": "foobar",
  "version": "1.0.0",
  "description": "",
  "main": "gulpfile.js",
  "scripts": {
    "build": "gulp build",
    "deploy": "gulp deploy",
    "buildAndDeploy": "gulp buildAndDeploy"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/paul-uz/XXX.git"
  },
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/paul-uz/XXX/issues"
  },
  "homepage": "https://github.com/paul-uz/XXX#readme",
  "dependencies": {
    "@aws-sdk/client-dynamodb": "^3.51.0",
    "@aws-sdk/client-s3": "^3.51.0",
    "@aws-sdk/lib-dynamodb": "^3.51.0",
    "@aws-sdk/util-dynamodb": "^3.51.0",
    "@mux/mux-node": "^4.0.0",
    "moment": "^2.29.1"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.92",
    "@types/node": "^14.18.3",
    "@typescript-eslint/eslint-plugin": "^5.12.1",
    "@typescript-eslint/parser": "^5.12.1",
    "custom-env": "^2.0.1",
    "del": "^6.0.0",
    "eslint": "^8.9.0",
    "glob": "^7.2.0",
    "gulp": "^4.0.2",
    "terser": "^5.11.0",
    "typescript": "^4.5.5"
  }
}

when I test the function, I get the following error:

{
  "errorType": "Error",
  "errorMessage": "Cannot find package '@aws-sdk/util-dynamodb' imported from /var/task/index.js\nDid you mean to import @aws-sdk/util-dynamodb/dist-cjs/index.js?",
  "trace": [
    "Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@aws-sdk/util-dynamodb' imported from /var/task/index.js",
    "Did you mean to import @aws-sdk/util-dynamodb/dist-cjs/index.js?",
    "    at new NodeError (internal/errors.js:322:7)",
    "    at packageResolve (internal/modules/esm/resolve.js:687:9)",
    "    at moduleResolve (internal/modules/esm/resolve.js:728:18)",
    "    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:842:11)",
    "    at Loader.resolve (internal/modules/esm/loader.js:89:40)",
    "    at Loader.getModuleJob (internal/modules/esm/loader.js:242:28)",
    "    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:76:40)",
    "    at link (internal/modules/esm/module_job.js:75:36)",
    "    at process.runNextTicks [as _tickCallback] (internal/process/task_queues.js:60:5)",
    "    at /var/runtime/deasync.js:23:15"
  ]
}

[paul-uz]

No one else had this issue?

[rnietoe]

same issue here: in my case it cannot find package 'aws-lambda-fastify'

i tried without success:

• path like .zip/nodejs/node_modules and .zip/nodejs/node14/node_modules
• archive the zip using windows or linux

[jcuna]

Having same issue

Cannot find package 'serverless-http' imported from /var/task/lambda.js

I only have that package in my lambda. It works without es module but unfortunately the app can only be built with ES module support due to external requirements.

[jerith666]

I've seen this error both importing other parts of my own source and importing individual source files from node_modules. It was caused by microsoft/TypeScript#40878, which is sadly marked Working as Intended.

I couldn't use the suggested work-around of adding .js extensions in the imports in my source code, because then Jest failed to run my tests.

Instead, I ended up post-processing the .js files that tsc produced to inject the .js extension into the imports, like so:

sed -E -i 's|from "./(.*)";?$|from "./\1.js";|' *.js

[djsjr]

any update on this? @RanVaknin

https://github.com/coderbyheart/aws-lambda-esm-with-layer
https://github.com/vibe/aws-esm-modules-layer-support

Here are two supposed "workarounds" I found. Yet to implement either

[paul-uz]

Would be good to get an update. What is the point in implementing ES module support when it doesn't work with Layers?!

Anyone got experience with the workarounds?

[boushib]

Any updates or fix for this issue?

[paul-uz]

Any updates or fix for this issue?

Kinda. Use Node 18. Otherwise, no, no official fix for other versions and I would say its unlikely, knowing AWS' track record with updates.

[serge7m]

You can get it working by specifying direct path to the imported file as /opt/nodejs/node_modules/xxx_your_package_xxx .
Here are the steps that will get you there:

// all of the below is the Lambda@Edge code that supports only Node16.
// 1. The below import stmt, copy-pasted from AWS docs, causes the "Cannot find package..." error in Lambda@Edge, apparently because current support of Lambda@Edge for NodeJS is limited to version 16.
//import { InvokeCommand, LambdaClient, LogType } from "@aws-sdk/client-lambda";

// 2. Determine where your zipped layer ends up in the Lambda environment under /opt, by uncommenting and iterating over this:
/*
console.log(-- NODE path: ${process.env.NODE_PATH}); // that's where you start it's /opt/nodejs and down
// list directories under NODE_PATH, descending into the subfolders until you discover your layer's packages
// modify the value of the "dir" as needed
import { readdirSync } from 'fs'
function getDirectories (dirPath) {
return readdirSync(dirPath, { withFileTypes: true })
.filter(dirent => dirent.isDirectory())
.map(dirent => ${dirent.name}\n );
}
dir = "/opt/nodejs/node_modules";
console.log (--${dir} contents: ${getDirectories(dir)} );
*/

// 3. Once you know where your layers reside under the /opt/..., use the fully qualified path to the imported file
import { InvokeCommand, LambdaClient, LogType } from "/opt/nodejs/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js";

[yenfryherrerafeliz]

Hi @paul-uz, @rnietoe, @jcuna, @jerith666, @djsjr,

Basically, this seems to be an issue at the lambda side, and for that I have opened an internal ticket with the lambda team.

In the meantime, while we wait for lambda team to answer, there is a workaround that we could use, which I found in the following post in stackoverflow, and is basically importing the package directly from the path. Actually what @serge7m says is correct.

Example:

import AWS from '/var/runtime/node_modules/aws-sdk/lib/aws.js';

So, a working example based on the sample code provide would be:

import AWS from '/var/runtime/node_modules/aws-sdk/lib/aws.js';

const {Lambda} = AWS;
const lambda = new Lambda();
export const handler = async (event) => {
    const eventSourceMappings = await lambda.listEventSourceMappings({ FunctionName: 'rpal-testlambda' }).promise();
    const response = {
        statusCode: 200,
        body: JSON.stringify(eventSourceMappings)
    };
    return response;
};

Thanks!

[trivikr]

The AWS SDK for JavaScript team revisited this issue while testing Lambda Layers, and verified that the symlink workaround from https://github.com/vibe/aws-esm-modules-layer-support works.

We recommend that workaround, as it's simpler.

[RanVaknin]

Since this is a limitation with Lambda and not actionable by the SDK team, I feel inclined to close this.

Thanks,
Ran~

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.