Skip to content

Add API to NpgsqlDataSource to resolve type mappings #6084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vonzshik wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add API to NpgsqlDataSource to resolve type mappings #6084

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Add API to NpgsqlDataSource to resolve type mappings
  • Loading branch information
@vonzshik
vonzshik committed Sep 10, 2025
commit d5314a837eda7d1a7721f643015f94ce9c7ffd99
63 changes: 63 additions & 0 deletions src/Npgsql/NpgsqlDataSource.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
using System.Transactions;
using Microsoft.Extensions.Logging;
using Npgsql.Internal;
using Npgsql.Internal.Postgres;
using Npgsql.Internal.ResolverFactories;
using Npgsql.Properties;
using Npgsql.Util;
Expand Down Expand Up @@ -247,6 +248,68 @@ public async Task ReloadTypesAsync(CancellationToken cancellationToken = default
}
}

/// <summary>
/// Attempts to return a mapping for a specific type.
/// </summary>
public PgTypeInfo? TryGetMapping<T>(string? dataTypeName = null) => TryGetMapping(typeof(T), dataTypeName);

/// <summary>
/// Attempts to return a mapping for a specific type.
/// </summary>
public PgTypeInfo? TryGetMapping(Type? type = null, string? dataTypeName = null)
Copy link
Copy Markdown
Member

@roji roji Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that PgTypeInfo is [Experimental], and also in the Internal namespace - I think that up to now the idea was that this was only for converter writers; we'd be exposing this to users for the first time. This also shows in the public API surface of this type - most of the stuff there probably shouldn't be shown to users.

I think that if we want to add an introspection API like this, we need another type. We do have the PostgresType hierarchy, but that really models the PG type only (no CLR type there, for example).

(just noting that there's no real urgent need for this API from my side)

/cc @NinoFloris

Copy link
Copy Markdown
Contributor Author

@vonzshik vonzshik Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if we want to add an introspection API like this, we need another type. We do have the PostgresType hierarchy, but that really models the PG type only (no CLR type there, for example).

Makes sense. We just have to agree on what exactly that type has to have. Probably, CLR type, the name of postgres's type + whether it supports writing?

Copy link
Copy Markdown
Member

@roji roji Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NpgsqlMappingInfo or something? Maybe @NinoFloris has a good idea here.

It could actually reference the PostgresType rather than just a string name (for full introspection capabilities). Besides that, just having the CLR type seems sufficient to me as a start, writability seems like an extra thing we don't necessarily need to do...

Copy link
Copy Markdown
Member

@roji roji Oct 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API design note: rather than a single method with two nullable parameters - which throws when both are null - we can have two methods, so that the annotations prevent the call at compile-time (internally they'd both just immediately call into a single method).

{
if (type is null && string.IsNullOrEmpty(dataTypeName))
throw new ArgumentException($"Either {nameof(type)} or {nameof(dataTypeName)} must be specified.");

// ReSharper disable once ConditionIsAlwaysTrueOrFalseAccordingToNullableAPIContract
if (SerializerOptions is null)
using (OpenConnection()) { }

return TryGetMappingCore(type, dataTypeName);
}

/// <summary>
/// Attempts to return a mapping for a specific type.
/// </summary>
public ValueTask<PgTypeInfo?> TryGetMappingAsync<T>(string? dataTypeName = null, CancellationToken cancellationToken = default)
Copy link
Copy Markdown
Member

@roji roji Oct 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that for cases where we don't actually need a generic type parameter, i.e. a Type parameter is enough (and will likely always be enough), there's no reason to introduce the additional generic API.

=> TryGetMappingAsync(typeof(T), dataTypeName, cancellationToken);

/// <summary>
/// Attempts to return a mapping for a specific type.
/// </summary>
public async ValueTask<PgTypeInfo?> TryGetMappingAsync(Type? type = null, string? dataTypeName = null, CancellationToken cancellationToken = default)
Copy link
Copy Markdown
Member

@roji roji Oct 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, ideally we'd be able to return mapping info to the user without ever connecting to the database, purely based on how the data source was configured - in which case we wouldn't need an async API here. This is a bit related to us returning a separate type (not PgTypeInfo), which would presumably return restricted info that's purely client-side mapping config.

Or am I missing something and this mapping information API really doesn't make sense without first connecting to the database and getting actual type OIDs etc.?

{
if (type is null && string.IsNullOrEmpty(dataTypeName))
throw new ArgumentException($"Either {nameof(type)} or {nameof(dataTypeName)} must be specified.");

// ReSharper disable once ConditionIsAlwaysTrueOrFalseAccordingToNullableAPIContract
if (SerializerOptions is null)
{
var connection = await OpenConnectionAsync(cancellationToken).ConfigureAwait(false);
await connection.DisposeAsync().ConfigureAwait(false);
}

return TryGetMappingCore(type, dataTypeName);
}

PgTypeInfo? TryGetMappingCore(Type? type, string? dataTypeName)
{
Debug.Assert(IsBootstrapped);
Debug.Assert(SerializerOptions is not null);
Debug.Assert(DatabaseInfo is not null);

PgTypeId? pgTypeID = null;

if (!string.IsNullOrEmpty(dataTypeName))
{
if (!DatabaseInfo.TryGetPostgresTypeByName(DataTypeName.NormalizeName(dataTypeName), out var pgType))
throw new NotSupportedException($"The data type name '{dataTypeName}' isn't present in your database. You may need to install an extension or upgrade to a newer version.");
pgTypeID = SerializerOptions.ToCanonicalTypeId(pgType.GetRepresentationalType());
}

return SerializerOptions.GetTypeInfoInternal(type, pgTypeID);
}

internal async Task Bootstrap(
NpgsqlConnector connector,
NpgsqlTimeout timeout,
Expand Down
4 changes: 4 additions & 0 deletions src/Npgsql/PublicAPI.Unshipped.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ Npgsql.NpgsqlConnectionStringBuilder.RequireAuth.get -> string?
Npgsql.NpgsqlConnectionStringBuilder.RequireAuth.set -> void
Npgsql.NpgsqlConnectionStringBuilder.SslNegotiation.get -> Npgsql.SslNegotiation
Npgsql.NpgsqlConnectionStringBuilder.SslNegotiation.set -> void
Npgsql.NpgsqlDataSource.TryGetMapping(System.Type? type = null, string? dataTypeName = null) -> Npgsql.Internal.PgTypeInfo?
Npgsql.NpgsqlDataSource.TryGetMapping<T>(string? dataTypeName = null) -> Npgsql.Internal.PgTypeInfo?
Npgsql.NpgsqlDataSource.TryGetMappingAsync(System.Type? type = null, string? dataTypeName = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask<Npgsql.Internal.PgTypeInfo?>
Npgsql.NpgsqlDataSource.TryGetMappingAsync<T>(string? dataTypeName = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.ValueTask<Npgsql.Internal.PgTypeInfo?>
Npgsql.NpgsqlDataSourceBuilder.ConfigureTypeLoading(System.Action<Npgsql.NpgsqlTypeLoadingOptionsBuilder!>! configureAction) -> Npgsql.NpgsqlDataSourceBuilder!
Npgsql.NpgsqlDataSourceBuilder.MapComposite(System.Type! clrType, string? pgName = null, Npgsql.INpgsqlNameTranslator? nameTranslator = null) -> Npgsql.NpgsqlDataSourceBuilder!
Npgsql.NpgsqlDataSourceBuilder.MapComposite<T>(string? pgName = null, Npgsql.INpgsqlNameTranslator? nameTranslator = null) -> Npgsql.NpgsqlDataSourceBuilder!
Expand Down
130 changes: 130 additions & 0 deletions test/Npgsql.Tests/DataSourceTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -448,5 +448,135 @@ public async Task ReloadTypes_across_data_sources([Values] bool async)
Assert.DoesNotThrowAsync(async () => await connection2.ExecuteScalarAsync($"SELECT 'happy'::{type}"));
}

[Test]
public async Task Resolve_type_mapping([Values] bool async)
{
var dataSource = DataSource;

var intClrTypeMapping = async
? await dataSource.TryGetMappingAsync<int>()
: dataSource.TryGetMapping<int>();

Assert.That(intClrTypeMapping, Is.Not.Null);
Assert.That(intClrTypeMapping!.Type, Is.EqualTo(typeof(int)));

var int4DataTypeNameMapping = async
? await dataSource.TryGetMappingAsync(dataTypeName: "int4")
: dataSource.TryGetMapping(dataTypeName: "int4");

Assert.That(int4DataTypeNameMapping, Is.Not.Null);
Assert.That(int4DataTypeNameMapping!.Type, Is.EqualTo(typeof(int)));
}

[Test]
public async Task Resolve_unknown_enum_type_mapping([Values] bool async)
{
await using var adminConnection = await OpenConnectionAsync();
var type = await GetTempTypeName(adminConnection);
await adminConnection.ExecuteNonQueryAsync($"CREATE TYPE {type} AS ENUM ('sad', 'ok', 'happy')");

var dataSource = DataSource;
// Reload types to load the new enum from the database
await dataSource.ReloadTypesAsync();

var enumClrTypeMapping = async
? await dataSource.TryGetMappingAsync<Mood>()
: dataSource.TryGetMapping<Mood>();

Assert.That(enumClrTypeMapping, Is.Null);

var enumDataTypeNameMapping = async
? await dataSource.TryGetMappingAsync(dataTypeName: type)
: dataSource.TryGetMapping(dataTypeName: type);

// We support mapping unknown enums to text
Assert.That(enumDataTypeNameMapping, Is.Not.Null);
Assert.That(enumDataTypeNameMapping!.Type, Is.EqualTo(typeof(string)));
}

[Test]
public async Task Resolve_registered_enum_type_mapping([Values] bool async)
{
await using var adminConnection = await OpenConnectionAsync();
var type = await GetTempTypeName(adminConnection);
await adminConnection.ExecuteNonQueryAsync($"CREATE TYPE {type} AS ENUM ('sad', 'ok', 'happy')");

await using var dataSource = CreateDataSource(dataSourceBuilder =>
{
dataSourceBuilder.MapEnum<Mood>(type);
});

var enumClrTypeMapping = async
? await dataSource.TryGetMappingAsync<Mood>()
: dataSource.TryGetMapping<Mood>();

Assert.That(enumClrTypeMapping, Is.Not.Null);
Assert.That(enumClrTypeMapping!.Type, Is.EqualTo(typeof(Mood)));

var enumDataTypeNameMapping = async
? await dataSource.TryGetMappingAsync(dataTypeName: type)
: dataSource.TryGetMapping(dataTypeName: type);

Assert.That(enumDataTypeNameMapping, Is.Not.Null);
Assert.That(enumDataTypeNameMapping!.Type, Is.EqualTo(typeof(Mood)));
}

[Test]
public async Task Resolve_unknown_composite_type_mapping([Values] bool async)
{
await using var adminConnection = await OpenConnectionAsync();
var type = await GetTempTypeName(adminConnection);
await adminConnection.ExecuteNonQueryAsync($"CREATE TYPE {type} AS (x int, some_text text)");

var dataSource = DataSource;
// Reload types to load the new enum from the database
await dataSource.ReloadTypesAsync();

var compositeClrTypeMapping = async
? await dataSource.TryGetMappingAsync<SomeComposite>()
: dataSource.TryGetMapping<SomeComposite>();

Assert.That(compositeClrTypeMapping, Is.Null);

var compositeDataTypeNameMapping = async
? await dataSource.TryGetMappingAsync(dataTypeName: type)
: dataSource.TryGetMapping(dataTypeName: type);

Assert.That(compositeDataTypeNameMapping, Is.Null);
}

[Test]
public async Task Resolve_registered_composite_type_mapping([Values] bool async)
{
await using var adminConnection = await OpenConnectionAsync();
var type = await GetTempTypeName(adminConnection);
await adminConnection.ExecuteNonQueryAsync($"CREATE TYPE {type} AS (x int, some_text text)");

await using var dataSource = CreateDataSource(dataSourceBuilder =>
{
dataSourceBuilder.MapComposite<SomeComposite>(type);
});

var compositeClrTypeMapping = async
? await dataSource.TryGetMappingAsync<SomeComposite>()
: dataSource.TryGetMapping<SomeComposite>();

Assert.That(compositeClrTypeMapping, Is.Not.Null);
Assert.That(compositeClrTypeMapping!.Type, Is.EqualTo(typeof(SomeComposite)));

var compositeDataTypeNameMapping = async
? await dataSource.TryGetMappingAsync(dataTypeName: type)
: dataSource.TryGetMapping(dataTypeName: type);

Assert.That(compositeDataTypeNameMapping, Is.Not.Null);
Assert.That(compositeDataTypeNameMapping!.Type, Is.EqualTo(typeof(SomeComposite)));
}

enum Mood { Sad, Ok, Happy }

record SomeComposite
{
public int X { get; set; }
public string SomeText { get; set; } = null!;
}
}