quarkus: vertx caching folder cannot be created with Amazon Lambda extension

Amazon Lambda extension requires an immutable file system.

Upon execution on AWS, vertx attempts to create the vertx cache folder. This causes the AWS Lambda execution to fail with the stack trace below. When run locally using sam-local (docker) it works ok.

The vertx configuration in io.quarkus.vertx.core.runtime.config.VertxConfiguration.caching is set to true by default. For the Amazon Lambda this should be set to false.

Execution on AWS fails:

java.lang.IllegalStateException: Failed to create cache dir
 at io.vertx.core.file.impl.FileResolver.setupCacheDir(FileResolver.java:332)
 at io.vertx.core.file.impl.FileResolver.<init>(FileResolver.java:87)
 at io.vertx.core.impl.VertxImpl.<init>(VertxImpl.java:165)
 at io.vertx.core.impl.VertxImpl.vertx(VertxImpl.java:92)
 at io.vertx.core.impl.VertxFactoryImpl.vertx(VertxFactoryImpl.java:40)
 at io.vertx.core.impl.VertxFactoryImpl.vertx(VertxFactoryImpl.java:32)
 at io.vertx.core.Vertx.vertx(Vertx.java:85)
 at io.quarkus.vertx.core.runtime.VertxCoreRecorder.initializeWeb(VertxCoreRecorder.java:100)
 at io.quarkus.vertx.core.runtime.VertxCoreRecorder.initializeWeb(VertxCoreRecorder.java:82)
 at io.quarkus.deployment.steps.VertxCoreProcessor$buildWeb104.deploy_0(VertxCoreProcessor$buildWeb104.zig:71)
 at io.quarkus.deployment.steps.VertxCoreProcessor$buildWeb104.deploy(VertxCoreProcessor$buildWeb104.zig:96)
 at io.quarkus.runner.ApplicationImpl5.doStart(ApplicationImpl5.zig:226)
 at io.quarkus.runtime.Application.start(Application.java:93)
 at io.quarkus.runtime.Application.run(Application.java:213)
 at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:34)

Code (Kotlin):

package org.tjpower

import com.amazonaws.services.lambda.runtime.Context
import com.amazonaws.services.lambda.runtime.RequestHandler
import javax.enterprise.context.ApplicationScoped
import javax.inject.Inject


class HelloRequest() {
    var firstName: String? = null
    var lastName: String? = null
}

@ApplicationScoped
open class HelloGreeter() {
    open fun greet(firstName: String?, lastName: String?): String {
        return "$firstName, $lastName"
    }
}

class HelloLambda : RequestHandler<HelloRequest, String> {

//    @Inject
//    lateinit var greeter: HelloGreeter

    val greeter = HelloGreeter()

    override fun handleRequest(request: HelloRequest, context: Context): String {
        return greeter.greet(request.firstName, request.lastName)
    }
}

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 30 (29 by maintainers)

Commits related to this issue

Issue #4254 Force java.io.tmpdir = /tmp for native executable to address vertx-cache folder creation problem — committed to oztimpower/quarkus by oztimpower 5 years ago
Merge pull request #4327 from oztimpower/master Issue #4254 Force java.io.tmpdir = /tmp for native executable — committed to quarkusio/quarkus by cescoffier 5 years ago

Most upvoted comments

That worked: *-runner -Djava.io.tmpdir=/tmp

However next problem, -H:EnableURLProtocols=http needs to be passed to the native-image command. For some reason that’s not appearing in the native-image command, captured below. When I manually added the http protocol to the native-image compile command, along with the above modification to java.io.tmpdir, everything worked ok, all fixed.

START RequestId: 211f3508-4af3-46ef-ab8a-73ce01946479 Version: $LATEST Exception in thread “Lambda Thread” 16:24:21 INFO [io.quarkus]] (main) Quarkus 999-SNAPSHOT started in 0.219s. Listening on: http://0.0.0.0:8080 com.oracle.svm.core.jdk.UnsupportedFeatureError: Accessing an URL protocol that was not enabled. The URL protocol http is supported but not enabled by default. It must be enabled by adding the -H:EnableURLProtocols=http option to the native-image command. 16:24:21 INFO [io.quarkus]] (main) Profile prod activated. at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:102) at com.oracle.svm.core.jdk.JavaNetSubstitutions.unsupported(JavaNetSubstitutions.java:196) 16:24:21 INFO [io.quarkus]] (main) Installed features: [cdi, kotlin, resteasy, resteasy-jsonb] at com.oracle.svm.core.jdk.JavaNetSubstitutions.getURLStreamHandler(JavaNetSubstitutions.java:157) at java.net.URL.getURLStreamHandler(URL.java:62) at java.net.URL.<init>(URL.java:599) at java.net.URL.<init>(URL.java:490) at java.net.URL.<init>(URL.java:439) at io.quarkus.amazon.lambda.runtime.AmazonLambdaApi.invocationNext(AmazonLambdaApi.java:31) at io.quarkus.amazon.lambda.runtime.AmazonLambdaRecorder$2.run(AmazonLambdaRecorder.java:59) at java.lang.Thread.run(Thread.java:748) at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:460) at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193) END RequestId: 211f3508-4af3-46ef-ab8a-73ce01946479 REPORT RequestId: 211f3508-4af3-46ef-ab8a-73ce01946479 Duration: 10010.17 ms Billed Duration: 10000 ms Memory Size: 128 MB Max Memory Used: 15 MB 2019-09-30T16:24:31.513Z 211f3508-4af3-46ef-ab8a-73ce01946479 Task timed out after 10.01 seconds

docker run -v /Users/timothypower/workspace/IdeaProjects/aws-samples/services/rest-kotlin/build:/project:z --rm quay.io/quarkus/ubi-quarkus-native-image:19.2.0.1 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dio.netty.noUnsafe=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dvertx.disableDnsResolver=true --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar rest-kotlin-0.0.1-SNAPSHOT-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:+PrintAnalysisCallTree -H:-AddAllCharsets -H:-SpawnIsolates -H:-JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace

On 30 Sep 2019, at 11:33 PM, Clement Escoffier notifications@github.com wrote:

Can you try to pass -Djava.io.tmpdir=/tmp to the native executable?

On 30 Sep 2019, at 16:51, oztimpower notifications@github.com wrote:

I have traced the issue. Yes /tmp is writeable for AWS Lambda, for both files and directories.

However native-image sees the java.io.tmpdir as “/var/tmp”, which is read-only. Is this determined at compile time in native-image by the docker linux container used?

The pure-java version works fine (which means it uses /tmp for it’s java.io http://java.io/.tmpdir on AWS lambda).

java.io http://java.io/.tmpdir appear to be different for native-image vs JVM.

I’ve updated the issue accordingly.

On 30 Sep 2019, at 1:52 PM, Clement Escoffier notifications@github.com wrote:

@geoand https://github.com/geoand the main impact would be when serving static files, as they are unpacked into the cache dir and read from there. In a lambda environment, serving static file is probably not used (@patriot1burke https://github.com/patriot1burke can you confirm?), so we could disable that.

Vert.x use the system tmp directory to create the cache directory, do it means that on AWS there is no tmp?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/quarkusio/quarkus/issues/4254?email_source=notifications&email_token=AIPGZO5XA6W2G6NS6RNNYNDQMGHZHA5CNFSM4I3N4VTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD74P4MQ#issuecomment-536411698, or mute the thread https://github.com/notifications/unsubscribe-auth/AIPGZOYG4PX6ZBFG6INR45TQMGHZHANCNFSM4I3N4VTA.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/quarkusio/quarkus/issues/4254?email_source=notifications&email_token=AADCG7JXD7ZDRW33NYL6LLLQMIHATA5CNFSM4I3N4VTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD755P6A#issuecomment-536598520, or mute the thread https://github.com/notifications/unsubscribe-auth/AADCG7MHFUI5OJRZGTCI7MDQMIHATANCNFSM4I3N4VTA.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/quarkusio/quarkus/issues/4254?email_source=notifications&email_token=AIPGZO342Y25CCV533WP233QMIL6HA5CNFSM4I3N4VTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD76CJRQ#issuecomment-536618182, or mute the thread https://github.com/notifications/unsubscribe-auth/AIPGZO3VG2MTOEC2Z64ZFGTQMIL6HANCNFSM4I3N4VTA.

oztimpower on Sep 30, 2019

@machi1990 https://github.com/quarkusio/quarkus/pull/5287

johnoliver on Nov 7, 2019

@johnoliver would you like to contribute the tip here https://github.com/quarkusio/quarkus/blob/9eb5164f24a2c3bc6b4015bc4e399a7ef955704c/docs/src/main/asciidoc/vertx.adoc ?

machi1990 on Nov 7, 2019

That would seem like a valuable addition to the vertx guide

geoand on Nov 7, 2019

for future reference for anyone who may stumble across this issue when deploying to openshift, I had to set -Dvertx.cacheDirBase=/tmp/vertx by creating an environment variable called JAVA_OPTS set to that value. Setting -Djava.io.tmpdir=/tmp alone did not seem to be sufficient.

johnoliver on Nov 7, 2019