liquibase: sqlFile + relativeToChangelogFile fails when using a logicalFilePath

Environment

Liquibase Version: 4.0.0

Liquibase Integration & Version: CLI

Liquibase Extension(s) & Version:

Database Vendor & Version: SQL SERVER 2019

Operating System Type & Version: WIN 10 Pro 2004 (19041.388)

Description

When a changelog has a ‘logicalFilePath’ set, execution of ‘sqlFile’ changes that have the ‘relativeToChangelogFile’ set to true fail with an IOException that says that the sql file can’t be found in a number of different paths. If ‘logicalFilePath’ is not set, everything works

Steps To Reproduce

Have a simple XML changelog that has an ‘logicalFilePath’ attribute in the root ‘databaseChangeLog’ node.
Have inside that changelog a changeset with a ‘sqlFile’ change that points to an existing file in the same folder than the changelog, and that has ‘relativeToChangelogFile=“true”’

<databaseChangeLog
    xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
    xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.6.xsd
    http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd"
    logicalFilePath="a-logical-file-path">
   

<changeSet id="some-changeset" author="NA">
      <sqlFile endDelimiter="GO"
        path="file.sql"
        relativeToChangelogFile="true"
       />
</changeSet> 
</databaseChangeLog>

Run liquibase update against the db

liquibase --driver=com.microsoft.sqlserver.jdbc.SQLServerDriver --url="jdbc:sqlserver://localhost:1433;database=the_database" --changeLogFile=changelog.xml  --username=the_username --password=thepassword --logLevel=debug update

Actual Behavior

Executions fails with an IOException.

Unexpected error running Liquibase: java.io.IOException: The file file.sql was not found in
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\access-bridge-64.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\cldrdata.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\dnsns.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\jaccess.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\localedata.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\nashorn.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\sunec.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\sunjce_provider.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\sunmscapi.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\sunpkcs11.jar
    - C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre\lib\ext\zipfs.jar

...

Specifying files by absolute path was removed in Liquibase 4.0. Please use a relative path or add '/' to the classpath parameter.
For more information, please use the --logLevel flag

Expected/Desired Behavior

No error happens and the sql file is executed against the database.

Additional Context

This is happening since 4.0.0 beta1. Version 3.10 works fine
Peeking a little inside the code, it seems that SQLFileChange.openSqlStream is using the filepath of the changeset (which contains the logicalFilePath) instead of the physical path of the containing changelog (which seems to be the behavior of v 3.x)

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 9
Comments: 35 (11 by maintainers)

Most upvoted comments

The bug is still effective on 4.6.1

+10

msimko81 on Dec 2, 2021

Hi! We are really blocked by the same issue and can’t upgrade liquibase to 4.x.x from 3.5.x. When you are going to release a patch with this fix? Thanks!

antenko on Jul 19, 2021

Hey @imod (and others) – we are working on a significant 4.4 release, and this has been sorted into the release after that. 4.4 is nearing dev-complete, but to get uat and test-complete will be a couple of weeks still.

we definitely hear that classpath, relative file path, and searchpath cause frictions for people so we need to look it holistically. thanks for the patience!

mariochampion on May 4, 2021

I debugged it and the problem is that searchPath in liquibase.integration.spring.SpringResourceAccessor#openStreams is created by concat of the source file and relative path of the referenced file, but should be concat of directory of the source file and the referenced file.

konikvranik on Jan 11, 2022

I was about two report a similar issue but it may be related to this one. My problem occurs when I use Liquibase in a Spring Boot project where changelog files are kept in the classpath “root” i.e. src/main/resources directory. Attempting to reference another file path as a relative one to the source file doesn’t work.

src/main/resources/main.xml:

(...)
 <sqlFile path="file.sql" relativeToChangelogFile="true" />
(...)

src/main/resources/file.sql:

select 1 from dual;

I managed to debug the issue the same way @konikvranik did: The problem’s source lies in SpringResourceAccessor@139:

searchPath = relativeTo.replaceFirst("/[^/]+$", "") + "/" + path;

The replaceFirst was probably meant to remove the filename from the path ( = obtain its directory) but it doesn’t work when the source path contains only a filename.

A similar implementation is present in other ResourceAccessor implementations e.g. ClassLoaderResourceAccessor@171 but additional checks are performed earlier to properly handle “slash-less” paths.

If using a subdirectory to keep changeset files is not a requirement (I haven’t found anything in the documentation) then I’d consider it a bug. A bug which could fixed quite easily by mirroring ClassLoaderResourceAccessor checks in SpringResourceAccessor (I’m ignoring the fact that perfrming path-related operations using regular expressions is always risky 😃).

If any maintainer could confirm that it should be possible to use src/main/resources directory to store changeset files in a Spring Boot app (without any subdirectories) that I’m happy to create a seaprate issue with a sample app to reproduce the issue and/or prepare a PR with a fix.

konradczajka on Feb 21, 2022

HI @sfxcode It should be set for the next couple of releases. We will try to pull it in earlier.

molivasdat on Dec 10, 2020

Sorry, it is #2060

smainz on Dec 17, 2021