Repository navigation
Lexbox API to create empty FLEx project #2439
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
71c590b
cbbb914
466212a
0638ee9
d43b45a
f156505
db9dd95
543004f
24e1bc6
5415444
b3f723f
6baffac
2374ea0
14fc50a
7d00f06
a31d137
1977e22
51b46b8
e01f766
94a74b3
666789e
2cf3b2d
abc9eee
9d1f536
1d14543
c0fc9cb
be9f3fa
fe0e27e
7dfe0ef
cfaf87c
f6c1c99
5b37c5d
0dc17b7
d746810
b7c75ad
b808a83
3cb5384
adabe5b
341bc34
d73ba66
04cf3fd
a7e203d
f9e036a
a81fced
f659ba7
cb2cc88
69176e0
f6f1fe7
19edb58
da2a31b
a457b8d
e3bd351
d2fe997
49c53f5
ad0f00a
a7579cc
da27256
0bfd8e2
02df235
c451523
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| using System.Diagnostics; | ||
| using FwHeadless.Services; | ||
| using LexCore.Entities; | ||
| using LexCore.Sync; | ||
| using Microsoft.AspNetCore.Http.HttpResults; | ||
| using Microsoft.Extensions.Options; | ||
| using SIL.WritingSystems; | ||
|
|
||
| namespace FwHeadless.Routes; | ||
|
|
||
| public static class ProjectRoutes | ||
| { | ||
| public static IEndpointConventionBuilder MapProjectRoutes(this WebApplication app) | ||
| { | ||
| var group = app.MapGroup("/api/project"); | ||
| group.MapPost("/initFwDataProject", InitFwDataProject); | ||
|
Comment on lines
+15
to
+16
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🛡️ Detected with Advanced Tier | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- local guidance ---'
find .. -name AGENTS.md -o -name AGENTS.local.md 2>/dev/null | sort | while read -r f; do
printf '\n### %s\n' "$f"
cat "$f"
done
printf '%s\n' '--- route outline ---'
ast-grep outline backend/FwHeadless/Routes/ProjectRoutes.cs
printf '%s\n' '--- route source ---'
cat -n backend/FwHeadless/Routes/ProjectRoutes.cs
printf '%s\n' '--- related symbols ---'
rg -n -S --glob '*.cs' --glob '*.csproj' 'MapGroup|MapPost|UseAuthentication|UseAuthorization|AddAuthentication|AddAuthorization|RequireAuthorization|ProjectCreationService|InitFwDataProject' backend/FwHeadless
printf '%s\n' '--- FwHeadless project files ---'
git ls-files 'backend/FwHeadless' | sed -n '1,160p'
printf '%s\n' '--- deployment references ---'
rg -n -S --glob '*.yml' --glob '*.yaml' --glob '*.json' --glob '*.cs' --glob '*.csproj' 'FwHeadless|fwheadless|api/project|initFwDataProject' . | sed -n '1,240p'Repository: sillsdev/languageforge-lexbox Length of output: 42548 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- applicable guidance ---'
for f in AGENTS.md backend/AGENTS.md backend/FwHeadless/AGENTS.md backend/FwHeadless/AGENTS.local.md; do
if [ -f "$f" ]; then
printf '\n### %s\n' "$f"
cat "$f"
fi
done
printf '%s\n' '--- ProjectRoutes.cs ---'
cat -n backend/FwHeadless/Routes/ProjectRoutes.cs
printf '%s\n' '--- Program.cs ---'
cat -n backend/FwHeadless/Program.cs
printf '%s\n' '--- FwHeadlessKernel auth-related registrations ---'
rg -n -C 5 -S 'AddAuthentication|AddAuthorization|Authentication|Authorization|RequireAuthorization|AddFwHeadless|HttpClient|LexboxUsername|LexboxPassword' backend/FwHeadless/FwHeadlessKernel.cs backend/FwHeadless backend/LexBoxApi --glob '*.cs' | sed -n '1,260p'
printf '%s\n' '--- FwHeadlessClient ---'
cat -n backend/LexBoxApi/Services/FwHeadlessClient.cs 2>/dev/null || true
printf '%s\n' '--- caller and middleware references ---'
rg -n -C 4 -S 'UseAuthentication|UseAuthorization|MapProjectRoutes|MapControllers|MapGroup|RequireAuthorization|AddAuthentication|AddAuthorization|JwtBearer|Bearer|Basic|service.?to.?service' backend --glob '*.cs' | sed -n '1,320p'Repository: sillsdev/languageforge-lexbox Length of output: 42044 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- route and project creation sink ---'
cat -n backend/FwHeadless/Routes/ProjectRoutes.cs
rg -n -S 'class ProjectCreationService|InitFwDataProject|Clone|Push|SendReceive|Mercurial|LexboxUsername|LexboxPassword' backend/FwHeadless/Services backend/FwHeadless/Routes --glob '*.cs' | sed -n '1,220p'
printf '%s\n' '--- bounded deployment files ---'
git ls-files deployment .github | rg -i 'fw.?headless|kustom|service|ingress|network.?policy|skaffold' | sed -n '1,220p'
printf '%s\n' '--- FwHeadless deployment/service exposure ---'
rg -n -C 5 -i 'fw-headless|fwheadless|kind: (service|ingress|networkpolicy)|type: (loadbalancer|nodeport|externalname)|ingress|networkpolicy' deployment .github --glob '*.yaml' --glob '*.yml' --glob '*.json' --glob '*.tmpl' 2>/dev/null | sed -n '1,320p'Repository: sillsdev/languageforge-lexbox Length of output: 36624 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- base FwHeadless deployment ---'
cat -n deployment/base/fw-headless-deployment.yaml
printf '%s\n' '--- base kustomization and ingress ---'
cat -n deployment/base/kustomization.yaml
cat -n deployment/base/ingress-config.yaml
printf '%s\n' '--- service and route definitions in deployment ---'
rg -n -C 8 -i 'kind: Service|name: fw-headless|serviceName|backend:|service:|paths:|/api|fwheadless' deployment/base deployment/develop deployment/staging deployment/production deployment/gha --glob '*.yaml' --glob '*.yml' | sed -n '1,360p'Repository: sillsdev/languageforge-lexbox Length of output: 29592 Broken Authentication Reachability: External Require service-to-service authentication for FwHeadless project initialization.
🤖 Prompt for AI Agents |
||
| group.MapGet("/creation-status", GetCreationStatus); | ||
| group.MapGet("/await-creation-finished", AwaitCreationFinished); | ||
| return group; | ||
| } | ||
|
|
||
| // Internal endpoint: LexBoxApi has already created the project row + empty repo and performed the | ||
| // admin-auth check. FwHeadless populates the empty repo with a template .fwdata configured for the | ||
| // requested writing systems. Not idempotent — on failure LexBoxApi deletes and recreates. | ||
| private static async Task<Results<Ok, ProblemHttpResult>> InitFwDataProject( | ||
| Guid projectId, | ||
| InitFwDataProjectInput input, | ||
| IProjectLookupService projectLookupService, | ||
| ProjectCreationService projectCreationService, | ||
| ILogger<Program> logger) | ||
| { | ||
| // Null-safe against a malformed body (System.Text.Json leaves an absent list null). | ||
| var wsError = InitFwDataProjectInput.ValidateWritingSystemList("vernacular", input.WsVernacular) | ||
| ?? InitFwDataProjectInput.ValidateWritingSystemList("analysis", input.WsAnalysis) | ||
| ?? InitFwDataProjectInput.ValidateUiWritingSystem(input.WsUi); | ||
| if (wsError is not null) | ||
| return TypedResults.Problem(wsError, statusCode: StatusCodes.Status400BadRequest); | ||
| var invalidWs = input.WsVernacular.Concat(input.WsAnalysis).FirstOrDefault(ws => !IetfLanguageTag.IsValid(ws)); | ||
| if (invalidWs is not null) | ||
| return TypedResults.Problem($"Invalid writing system code: {invalidWs}", | ||
| statusCode: StatusCodes.Status400BadRequest); | ||
| if (!IetfLanguageTag.IsValid(input.WsUi)) | ||
| return TypedResults.Problem($"Invalid UI writing system code: {input.WsUi}", | ||
| statusCode: StatusCodes.Status400BadRequest); | ||
|
|
||
| var projectCode = await projectLookupService.GetProjectCode(projectId); | ||
| if (projectCode is null) | ||
| { | ||
| logger.LogError("initFwDataProject request for non-existent project {ProjectId}", projectId); | ||
| return TypedResults.Problem("Project not found", statusCode: StatusCodes.Status404NotFound); | ||
| } | ||
|
|
||
| await projectCreationService.InitFwDataProject( | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this will throw and result in a 500 if creation is in progress. I'm not sure that's correct. Also what happens if the project already exists (from any source) and this api is called? that seems like undefined behavior.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I need to look at this again in light of commit 02df235, to see if its changes have influenced this. (E.g., I don't think this will result in a 500 anymore, but I need to double-check). |
||
| projectId, projectCode, input.WsVernacular, input.WsAnalysis, input.WsUi); | ||
| return TypedResults.Ok(); | ||
| } | ||
|
|
||
| // Instantaneous creation status, for a caller that wants to poll. Mirrors /api/merge/status. | ||
| private static async Task<Results<Ok<ProjectCreationStatus>, NotFound>> GetCreationStatus( | ||
| Guid projectId, | ||
| SyncHostedService syncHostedService, | ||
| IProjectLookupService projectLookupService, | ||
| IOptions<FwHeadlessConfig> config) | ||
| { | ||
| using var activity = FwHeadlessActivitySource.Value.StartActivity(); | ||
| activity?.SetTag("app.project_id", projectId); | ||
|
|
||
| var status = await DetermineCreationStatus(projectId, syncHostedService, projectLookupService, config); | ||
| if (status is null) | ||
| { | ||
| activity?.SetStatus(ActivityStatusCode.Error, "Project not found"); | ||
| return TypedResults.NotFound(); | ||
| } | ||
| activity?.SetStatus(ActivityStatusCode.Ok, status.Status.ToString()); | ||
| return TypedResults.Ok(status); | ||
| } | ||
|
|
||
| // Long-poll until an in-flight creation finishes. Mirrors /api/merge/await-finished: giving up on the | ||
| // wait is reported as a status, not an error, because the creation itself keeps running either way. | ||
| private static async Task<Results<Ok<ProjectCreationStatus>, NotFound>> AwaitCreationFinished( | ||
| Guid projectId, | ||
| SyncHostedService syncHostedService, | ||
| IProjectLookupService projectLookupService, | ||
| IOptions<FwHeadlessConfig> config, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| using var activity = FwHeadlessActivitySource.Value.StartActivity(); | ||
| activity?.SetTag("app.project_id", projectId); | ||
|
|
||
| try | ||
| { | ||
| var result = await syncHostedService.AwaitCreationFinished(projectId, cancellationToken); | ||
| if (result is not null) | ||
| { | ||
| activity?.SetStatus(ActivityStatusCode.Ok, result.Status.ToString()); | ||
| return TypedResults.Ok(result); | ||
| } | ||
| } | ||
| catch (OperationCanceledException) | ||
| { | ||
| // The caller's wait was cancelled; the creation was not (it takes no cancellation token). | ||
| // Report that distinctly so a caller can't read a timeout as a failed creation. | ||
| activity?.SetStatus(ActivityStatusCode.Unset, "Timed out awaiting creation"); | ||
| return TypedResults.Ok(ProjectCreationStatus.TimedOutAwaitingCreation); | ||
| } | ||
|
|
||
| // Nothing in flight and nothing remembered, so fall back to what the project itself tells us. | ||
| var status = await DetermineCreationStatus(projectId, syncHostedService, projectLookupService, config); | ||
| if (status is null) | ||
| { | ||
| activity?.SetStatus(ActivityStatusCode.Error, "Project not found"); | ||
| return TypedResults.NotFound(); | ||
| } | ||
| activity?.SetStatus(ActivityStatusCode.Ok, status.Status.ToString()); | ||
| return TypedResults.Ok(status); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Creation status for a project, or null when LexBox has no such project. In-memory state wins | ||
| /// when we have it; otherwise the status is derived from durable state, so it survives a FwHeadless | ||
| /// restart and a result that has aged out of the cache. | ||
| /// </summary> | ||
| private static async Task<ProjectCreationStatus?> DetermineCreationStatus( | ||
| Guid projectId, | ||
| SyncHostedService syncHostedService, | ||
| IProjectLookupService projectLookupService, | ||
| IOptions<FwHeadlessConfig> config) | ||
| { | ||
| if (syncHostedService.IsProjectBeingCreated(projectId)) return ProjectCreationStatus.Creating; | ||
| var recent = syncHostedService.TryGetRecentCreationResult(projectId); | ||
| if (recent is not null) return recent; | ||
|
|
||
| var projectCode = await projectLookupService.GetProjectCode(projectId); | ||
| if (projectCode is null) return null; | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what does a null creation status mean? could we just return unknown or not found?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
||
| // A successful creation leaves the .fwdata on disk; a failed one deletes the project folder | ||
| // (ProjectCreationService.CleanupLocalProject). Note this says "project data is here", not "this | ||
| // process created it" -- a project cloned by an ordinary sync looks the same, which is the right | ||
| // answer for a caller asking whether the project is ready to use. | ||
| var fwDataProject = config.Value.GetFwDataProject(projectCode, projectId); | ||
| return File.Exists(fwDataProject.FilePath) | ||
| ? ProjectCreationStatus.Created | ||
| : ProjectCreationStatus.NotCreated; | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I say we just make that constant public, I don't see a reason why not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense to me, but that will require getting a FLExBridge PR pushed through, which might take some time. Let's keep this constant until the FLExBridge PR is approved and merged, then once we are able to upgrade to a new FLExBridge version we can reference its public constant instead of the hardcoded "7500002" string. But I don't want to hold up merging this feature long enough to get a FLExBridge PR approved.