Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ITranslationProvider.LoadTranslationsAsync() #15886

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open

Add ITranslationProvider.LoadTranslationsAsync() #15886

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
11d28ed
Add ITranslationProvider.LoadTranslationsAsync()
hishamco Apr 26, 2024
62a0e0b
IPluralStringLocalizer.GetTranslationAsync()
hishamco Apr 26, 2024
f8246fd
Fix the build
hishamco Apr 26, 2024
0041fbf
Apply suggestions from code review
hishamco Apr 26, 2024
ee4a668
Address feedback
hishamco Apr 26, 2024
93a802e
Use IAsyncEnumerable
hishamco Apr 26, 2024
3035a04
Revert "Use IAsyncEnumerable"
hishamco Apr 26, 2024
239bc7f
Merge branch 'main' into hishamco/translation-provider
hishamco May 2, 2024
5cd5a85
Merge branch 'main' into hishamco/translation-provider
hishamco May 3, 2024
8fdfc09
Fix merge conflict
hishamco May 3, 2024
47050eb
GetAllStringsFromCultureHierarchyAsync() return IAsyncEnumerable<Loca…
hishamco May 3, 2024
1406101
Merge branch 'main' into hishamco/translation-provider
hishamco Aug 15, 2024
431cb66
Merge branch 'main' into hishamco/translation-provider
hishamco Aug 15, 2024
7a3b976
Merge branch 'main' into hishamco/translation-provider
hishamco Dec 13, 2024
89330e1
Update
hishamco Dec 14, 2024
bebc0af
Merge branch 'main' into hishamco/translation-provider
hishamco Dec 14, 2024
b21f7e9
Merge branch 'main' into hishamco/translation-provider
hishamco Jan 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address feedback
  • Loading branch information
@hishamco
hishamco committed Apr 26, 2024
commit ee4a6685c6a1df685fe7d87de68b3d16a8b9c691
8 changes: 3 additions & 5 deletions src/OrchardCore/OrchardCore.Localization.Core/LocalizationManager.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Caching.Memory;

Expand Down Expand Up @@ -40,7 +38,7 @@ public LocalizationManager(
/// <inheritdoc />
public async Task<CultureDictionary> GetDictionaryAsync(CultureInfo culture)
{
var cachedDictionary = _cache.GetOrCreate(CacheKeyPrefix + culture.Name, k => new Lazy<Task<CultureDictionary>>(async () =>
var cachedDictionary = await _cache.GetOrCreateAsync(CacheKeyPrefix + culture.Name, async (e) =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sicne we don"t want entries to be revoked here (we don"t, right? even on memory pressure), we can use a keyed Dictionary and remove the prefix on the key (because there can"t be conflicts anymore). I did it recently, I am sure you will recall.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use ConccurentDictionary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, a ConcurrentDictionary actually since a culture could be read while another one is being added to it. But still a keyed service so we don"t have to use IMemoryCache. We should only use it when we need expiration rules which we don"t want here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add a comment in the code to remember why we need a ConcurrentDictionary (I will forget myself)

{
var rule = _defaultPluralRule;

Expand All @@ -59,8 +57,8 @@ public async Task<CultureDictionary> GetDictionaryAsync(CultureInfo culture)
}

return dictionary;
}, LazyThreadSafetyMode.ExecutionAndPublication));
});

return await cachedDictionary.Value;
return cachedDictionary;
}
}
3 changes: 2 additions & 1 deletion src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoFilesTranslationsProvider.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.Extensions.FileProviders;

Expand Down Expand Up @@ -37,7 +38,7 @@ private async Task LoadFileToDictionaryAsync(IFileInfo fileInfo, CultureDictiona
{
using var stream = fileInfo.CreateReadStream();
using var reader = new StreamReader(stream);
dictionary.MergeTranslations(await _parser.ParseAsync(reader));
dictionary.MergeTranslations(await _parser.ParseAsync(reader).ToListAsync());
}
}
}
13 changes: 5 additions & 8 deletions src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,19 @@ public class PoParser
/// </summary>
/// <param name="reader">The <see cref="TextReader"/>.</param>
/// <returns>A list of culture records.</returns>
[Obsolete("This method has been deprecated, please use ParseAsync instead.")]
public IEnumerable<CultureDictionaryRecord> Parse(TextReader reader) => ParseAsync(reader).GetAwaiter().GetResult();
[Obsolete("This methos has been deprecated, please use ParseAsync instead.")]
hishamco marked this conversation as resolved.
Show resolved Hide resolved
public IEnumerable<CultureDictionaryRecord> Parse(TextReader reader) => ParseAsync(reader).ToEnumerable();

/// <summary>
/// Parses a .po file.
/// </summary>
/// <param name="reader">The <see cref="TextReader"/>.</param>
/// <returns>A list of culture records.</returns>
#pragma warning disable CA1822 // Mark members as static
public async Task<IEnumerable<CultureDictionaryRecord>> ParseAsync(TextReader reader)
public async IAsyncEnumerable<CultureDictionaryRecord> ParseAsync(TextReader reader)
#pragma warning restore CA1822 // Mark members as static
{
var entryBuilder = new DictionaryRecordBuilder();
var cultureDictionaryRecords = new List<CultureDictionaryRecord>();
string line;
while ((line = await reader.ReadLineAsync()) != null)
{
Expand All @@ -51,18 +50,16 @@ public async Task<IEnumerable<CultureDictionaryRecord>> ParseAsync(TextReader re
// msgid or msgctxt are first lines of the entry. If builder contains valid entry return it and start building a new one.
if ((context == PoContext.MessageId || context == PoContext.MessageContext) && entryBuilder.ShouldFlushRecord)
{
cultureDictionaryRecords.Add(entryBuilder.BuildRecordAndReset());
yield return entryBuilder.BuildRecordAndReset();
}

entryBuilder.Set(context, content);
}

if (entryBuilder.ShouldFlushRecord)
{
cultureDictionaryRecords.Add(entryBuilder.BuildRecordAndReset());
yield return entryBuilder.BuildRecordAndReset();
}

return cultureDictionaryRecords;
}

private static string Unescape(string str)
Expand Down
2 changes: 1 addition & 1 deletion test/OrchardCore.Tests/Localization/PoParserTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ private static async Task<CultureDictionaryRecord[]> ParseTextAsync(string resou
using var resource = testAssembly.GetManifestResourceStream("OrchardCore.Tests.Localization.PoFiles." + resourceName + ".po");
using var reader = new StreamReader(resource);

return (await parser.ParseAsync(reader)).ToArray();
return (await parser.ParseAsync(reader).ToListAsync()).ToArray();
}
}
}