SQLitePCL.raw: Unexpected System.AccessViolationException when using Microsoft.Data.Sqlite
Hi, I’m using this library in my personal project and when executing a query, there is a probability that a System.AccessViolationException will occur.
I am sorry that I cannot find the cause of the problem. I came here for some help
The error log is as follows:
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at SQLitePCL.SQLite3Provider_dynamic_cdecl.SQLitePCL.ISQLite3Provider.sqlite3_prepare_v2(SQLitePCL.sqlite3, System.ReadOnlySpan`1<Byte>, IntPtr ByRef, System.ReadOnlySpan`1<Byte> ByRef)
at Microsoft.Data.Sqlite.SqliteCommand+<PrepareAndEnumerateStatements>d__64.MoveNext()
at Microsoft.Data.Sqlite.SqliteCommand+<GetStatements>d__54.MoveNext()
at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(System.Data.CommandBehavior)
at ConsoleApp1.Program.Main(System.String[])
I have stable reproduction code as follows: (Usually runs will crash within 30 seconds)
using System;
using System.Data;
using Microsoft.Data.Sqlite;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
using (var connection =
new SqliteConnection(
@"Data Source=C:\Users\eyhn\Projects\Anything\core\Anything\bin\Debug\net5.0\cache\tracker.db"))
{
connection.Open();
while (true)
{
using (connection.BeginTransaction())
{
var queryCommand = connection.CreateCommand();
queryCommand.CommandText = $@"
SELECT Id, Url, Parent, IsDirectory, IdentifierTag, ContentTag FROM FileTracker WHERE Url=?1;
";
queryCommand.Parameters.Add(new SqliteParameter("?1", "file://local/"));
var reader = queryCommand.ExecuteReader();
var result = HandleReaderSingleDataRow(reader);
Console.WriteLine(result?.Id + "");
}
}
}
}
private static DataRow HandleReaderSingleDataRow(IDataReader reader)
{
if (!reader.Read())
{
return null;
}
return new DataRow(
reader.GetInt64(0),
reader.GetString(1),
reader.GetValue(2) as long?,
reader.GetBoolean(3),
reader.GetValue(4) as string,
reader.GetValue(5) as string);
}
public record DataRow(long Id, string Url, long? Parent, bool IsDirectory, string IdentifierTag,
string ContentTag);
}
}
The package version i used:
> dotnet list package --include-transitive
Project 'ConsoleApp1' has the following package references
[net5.0]:
Top-level Package Requested Resolved
> Microsoft.Data.Sqlite 6.0.0-preview.6.21352.1 6.0.0-preview.6.21352.1
Transitive Package Resolved
> Microsoft.Data.Sqlite.Core 6.0.0-preview.6.21352.1
> SQLitePCLRaw.bundle_e_sqlite3 2.0.5-pre20210521085756
> SQLitePCLRaw.core 2.0.5-pre20210521085756
> SQLitePCLRaw.lib.e_sqlite3 2.0.5-pre20210521085756
> SQLitePCLRaw.provider.dynamic_cdecl 2.0.5-pre20210521085756
> System.Memory 4.5.3
After my attempts, the same problem appeared on windows pc and macbook.
I’ll upload the database file I’m using, and this file seems to be the only one that’s having problems.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 3
- Comments: 42 (35 by maintainers)
Commits related to this issue
- repro stuff for #430 — committed to ericsink/SQLitePCL.raw by ericsink 3 years ago
- undo the debug prints in provider.tt from the prev commit. change the #430 repro to use ugly instead of Microsoft.Data.Sqlite. — committed to ericsink/SQLitePCL.raw by ericsink 3 years ago
- various changes to the #430 repro to make it more like the Microsoft.Data.Sqlite counterpart. still can't make it crash. — committed to ericsink/SQLitePCL.raw by ericsink 3 years ago
- bring back original #430 repro so I can run either one. bug430_orig uses Microsoft.Data.Sqlite. bug430_ugly is my attempt to repro the same bug without Microsoft.Data.Sqlite. — committed to ericsink/SQLitePCL.raw by ericsink 3 years ago
- SQLite: Revert connection pool PR Since merging, the CI fail rate increased significantly from ericsink/SQLitePCL.raw#430 Fixes #16202, unresolves #13837 — committed to bricelam/efcore by bricelam 3 years ago
- SQLite: Revert connection pool PR Since merging, the CI fail rate increased significantly from ericsink/SQLitePCL.raw#430 Fixes #16202, unresolves #13837 — committed to bricelam/efcore by bricelam 3 years ago
- in the provider api, rm the utf8z overloads for sqlite3_prepare_v2/v3. these overloads were not memory safe, as they returned a span made from a pointer obtained from a fixed block. technically, thi... — committed to ericsink/SQLitePCL.raw by ericsink 3 years ago
- Revert "in the provider api, rm the utf8z overloads for sqlite3_prepare_v2/v3. these overloads were not memory safe, as they returned a span made from a pointer obtained from a fixed block. technica... — committed to ericsink/SQLitePCL.raw by ericsink 3 years ago
Hi,
I just randomly stumbled across this issue. I think it hasn’t been noted yet, but to me it seems clear why exceptions like
AccessViolationException
can happen using the repro code in https://github.com/ericsink/SQLitePCL.raw/issues/430#issuecomment-899733647, whenSQLITE_OPEN_NOMUTEX
is specified for the connection (which is not a bug in SQLite or the GC/CLR):Notice that the code creates a new
SqliteCommand
command in a loop without ausing
statement (or calling.Dispose()
on it). This means that some time after theSqliteCommand
is no longer referenced, the GC will eventually call the finalizer (in a separate thread) of this object, which callsDispose(false)
(since it hasn’t been disposed yet).SqliteCommand.Dispose(bool)
looks like this:Note it calls
DisposePreparedStatements
, passing thedisposing
parameter. Looking at this method:We can see that in the second
if
block, it iterates through the_preparedStatements
and calls.Dispose()
on them, regardless of whetherdisposing
istrue
orfalse
. (AFAIK this is not a correct implementation of the Dispose pattern, because when the finalizer is executed, object references on the current object might no longer be valid, and therefore the_preparedStatements
shouldn’t be accessed whendisposing
isfalse
).sqlite3_stmt.Dispose()
in turn calls itsReleaseHandle()
, which calls the SQLite native functionsqlite3_finalize()
in order to finalize the prepared statement.Therefore, it can happen that the finalizer thread calls the native
sqlite3_finalize()
function for a specific native DB connection at the same time when the main thread calls other SQLite functions likesqlite3_prepare_v2()
(withinCreateCommand()
/ExecuteScalar()
) for the same native DB connection, which is not allowed when usingSQLITE_OPEN_NOMUTEX
, because then each DB connection may only be accessed by a single thread at the same time.(Note: Even if the
SqliteCommand.DisposePreparedStatements()
is changed to only iterate over_preparedStatements
whendisposing
istrue
, the access violation will probably still occur because then the GC will eventually finalize thesqlite3_stmt
separately.)Thanks!
Thanks for all your help on this, @ericsink!
I’m seeing this error a lot more consistently in my pooling branch (https://github.com/dotnet/efcore/pull/25018). Like you, I don’t see anything obviously wrong with the Span code…
Yes, this was deliberately fixed on our side for RC1.
The stack trace is exactly the same going into sqlite3_prepare_v2 where the exception is thrown. Both throw AccessViolationException.