diff --git a/cmf-codereview/README.md b/cmf-codereview/README.md index e80b7f6..ef05bb6 100644 --- a/cmf-codereview/README.md +++ b/cmf-codereview/README.md @@ -60,7 +60,9 @@ documented in [docs/commands.md](docs/commands.md) — start there for anything ## What it looks for -See [docs/rules.md](docs/rules.md) for the full list of the 17 rules, their severity, and suggested fixes. +See [docs/rules.md](docs/rules.md) for checks, severities, and suggested fixes. +[PR validation](docs/pr-validation.md) covers DEE UseReference declarations, deployment folder +versions, and the added/renamed file input required by PR-Changes. ## Configuration diff --git a/cmf-codereview/docs/commands.md b/cmf-codereview/docs/commands.md index 85882ea..cf8d33e 100644 --- a/cmf-codereview/docs/commands.md +++ b/cmf-codereview/docs/commands.md @@ -14,7 +14,7 @@ Runs the review engine over a repository and prints findings as a JSON array by SARIF 2.1.0 log with `--report-format Sarif`. Optionally also saves the selected output to a file. ```bash -cmf-codereview scan --repository-root [--scope DEEs|Orchestration|Both] [--roots ...] [--output ] [--report-format Json|Sarif] [--changed-files-list ] [--fail-on Critical|High|Medium] [--update-baseline] +cmf-codereview scan --repository-root [--scope DEEs|Orchestration|Both] [--roots ...] [--output ] [--report-format Json|Sarif] [--changed-files-list ] [--added-files-list ] [--fail-on Critical|High|Medium] [--update-baseline] ``` | Option | Required | Default | Description | @@ -25,6 +25,7 @@ cmf-codereview scan --repository-root [--scope DEEs|Orchestration|Both] [ | `--output` | No | (none — stdout only) | Path to also write findings in the selected format to disk. Parent directories are created automatically. | | `--report-format` | No | `Json` | `Json` (findings array) or `Sarif` (SARIF 2.1.0). Values are case-insensitive. The selected format applies to stdout and `--output`. | | `--changed-files-list` | No | (none — reports on every scanned file) | Path to a text file listing changed files, one relative or absolute path per line (e.g. the output of `git diff --name-only`). When given, only findings on those files are returned — but the whole repository is still analyzed for cross-reference and loop-exposure context, so a finding on a changed file caused by an unchanged caller elsewhere is still correctly reported. Intended for CI gates that should only comment on files actually touched by a change. | +| `--added-files-list` | No | (deployment version check skipped) | Added/renamed PR destinations, one path per line. Enables package-version folder checks for new MasterData and Process Rules, including JSON. Use `git diff --name-only --diff-filter=AR --find-renames` with the same range as the changed-file list. | | `--fail-on` | No | (none — always exits `0`) | `Critical`, `High`, or `Medium`. When given, the command exits `1` if any reported finding is at or above this severity. Omit it to keep `scan` purely informational. | | `--update-baseline` | No | `false` | Write all current findings (ignoring any existing baseline) to `.codereview.baseline.json` at the repository root, accepting them as known debt — subsequent scans only report findings not in the baseline. Always exits `0` (ignores `--fail-on`). See [docs/configuration.md](configuration.md). | @@ -40,7 +41,7 @@ metadata the CM Framework build/pack tooling reads: sibling subfolders as Orchestration roots. - Every manifest with `"packageType": "Data"` contributes a DEE root for each `contentToPack` entry whose `"contentType": "DEE"`, resolved from that entry's own `source` glob (e.g. `"DEEs/*"` under - the manifest's directory) — so both the nested Micron-style `Cmf..Data.Packages/DEEs` shape + the manifest's directory) — so both the nested `Cmf..Data.Packages/DEEs` shape and a flat `Cmf..Data/DEEs` shape resolve correctly, and a repository with more than one Data package (e.g. a Tests package bundling its own DEE fixtures) gets every one of them. @@ -219,13 +220,19 @@ before rendering, so choosing SARIF does not change the findings or exit-code po This functionality writes report files. Publishing results or creating PR annotations requires a separately configured consumer; no pipeline generation or scaffolding is performed. +## PR deployment validation + +See [pr-validation.md](pr-validation.md) for DEE reference rules, package-version folder checks, +and the required Bash PR-Changes task edits. Historical folder versions are checked only for +explicit added/renamed files; the reviewer does not infer additions from the changed-file list. + ## Reviewing Process Rules Process Rule diagnostics run automatically as part of the DEEs/Both scope. No guideline file -or additional validation flag is required. To focus on the Micron deployment rules: +or additional validation flag is required. To focus on a customization project's deployment rules: ```powershell -cmf-codereview scan --repository-root C:\workspaces\Micron --roots Cmf.Custom.Data/Cmf.Custom.Data.Packages/DEEs/ProcessRules --fail-on High --output findings.json +cmf-codereview scan --repository-root C:\workspaces\Project --roots Cmf.Custom.Data/Cmf.Custom.Data.Packages/DEEs/ProcessRules --fail-on High --output findings.json cmf-codereview report --findings-path findings.json --report-mode Detailed --report-format Html --report-path process-rules.html ``` diff --git a/cmf-codereview/docs/configuration.md b/cmf-codereview/docs/configuration.md index 8bb1547..3130136 100644 --- a/cmf-codereview/docs/configuration.md +++ b/cmf-codereview/docs/configuration.md @@ -173,3 +173,11 @@ Those are the two supported kinds; a pattern that fits neither needs a code rule `docs/rules.md` and `IReviewRule`) — or ask for the kind to be added. Commit `.codereview.json` alongside the code so every teammate and CI run enforces the same rules. + +## DEE references and deployment versions + +Use the normal `rules` dictionary to configure `DeeInvalidReference`, `DeeMissingReference`, +`DeeReferenceUnresolved`, `DeploymentVersionMismatch`, and `DeploymentVersionUnresolved`. +For example, `"DeeReferenceUnresolved": { "enabled": false }` disables only unresolved-reference +review findings; it does not disable confirmed invalid or missing-reference findings. +See [pr-validation.md](pr-validation.md) for the metadata and added-file inputs needed for validation. diff --git a/cmf-codereview/docs/pr-validation.md b/cmf-codereview/docs/pr-validation.md new file mode 100644 index 0000000..153ccdb --- /dev/null +++ b/cmf-codereview/docs/pr-validation.md @@ -0,0 +1,151 @@ +# PR deployment and DEE reference validation + +The reviewer validates literal `UseReference(assembly, namespace)` declarations in `DeeDevBase` +methods and checks new deployment content against its owning Data package version. + +## DEE references + +Restore the DEE project and its project references before the scan. The reviewer reads the owning +`.csproj`, `obj/project.assets.json`, reference HintPaths and referenced project sources. It uses +Roslyn to bind type/extension usages and reads the referenced CMF compiler's default namespace and +assembly arrays from metadata. No product assemblies or type initializers are executed. + +- `DeeInvalidReference` (High): a declared namespace is known in the available metadata but is + unavailable from the combined default and explicitly declared assemblies. +- `DeeMissingReference` (High): a bound type/extension needs an import or assembly not supplied by + either a declaration or the compiler defaults. The finding points to the usage. +- `DeeReferenceUnresolved` (Medium): project/dependency metadata, compiler defaults, a type, an + assembly or non-literal arguments cannot be resolved. This is a validation limitation, not proof + that the reference is invalid. Unknown compiler initializer layouts are not executed or guessed. + +An unused C# `using` alone does not require a declaration. The product compiler combines the assembly +arguments independently of the namespace arguments: a namespace may be supplied by any default or +explicitly declared assembly, including a declaration that appears later. Empty assembly arguments, +assembly-only references, fully qualified type usages, and the harmless `UseReference("", "")` no-op +are supported. Framework assemblies supplied by the runtime take precedence over older NuGet facade +assets when binding symbols. The check covers direct `DeeDevBase` classes and deployed +`DeeActionCode`/`DeeTestCondition` methods; it does not evaluate MSBuild conditions, generate code, +execute helpers or prove completeness for arbitrary dynamic/alias-based imports. + +### Deferred Product parsing parity + +The reviewer currently inspects complete method bodies rather than extracting only the regions +between deployed-code markers. Its C# syntax analysis also accepts declaration forms that Product's +regex extraction does not, such as whitespace between `UseReference` and `(`. Declarations outside +the deployed region can incorrectly satisfy a reference, and syntax differences can cause missed +findings or false positives. Marker extraction, declaration-syntax parity and host configuration +remain deferred follow-up work. + +### Product compiler verification + +The product implementation in `Platform/Host/Biz/Core/Common/DynamicExecutionEngine/Compiler.cs` +was inspected at Product commit `306f9c47cb0`. `GenerateFullActionCode` creates a new class from the +action and condition bodies. `GetUsings` combines default imports with the second arguments of +literal declarations; `GetReferences` independently combines default references with their first +arguments. `GetDefaultReferences` resolves framework DLLs from the .NET runtime directory. +`CompileActionCode` additionally applies host-configured `alsoInclude` reference rules. +`RuleEvaluatorAdapter.UseReference` itself has an empty body. + +The reviewer does not reproduce a running host's configuration, keyword substitutions, arbitrary +reference paths or `alsoInclude` rules. It uses the dependencies restored in the reviewed workspace +and cannot prove that these match a deployed host. Core-only hosting and unsupported compiler +layouts also need separate validation; the current default import check targets MES projects. + +An opt-in check exercises the **actual compiler binary restored by a DEE project**, rather than the +unit tests' compiler fixture: + +```powershell +dotnet run --project tools/DeeCompilerCheck -c Release -- ` + C:\workspaces\Project\Cmf.Custom.Data\Cmf.Custom.Data.Packages\DEEs\Cmf.Custom.Actions.csproj ` + compiler-check.json +``` + +Restore that project first. This helper invokes `GenerateFullActionCode`, `GetReferences` and +`CompileSources`, resolving returned DLL names to the runtime and restored package files instead +of a deployed BusinessTier directory. It verifies missing assembly/import failures, the corrected +reference, independent inputs, and the empty no-op. It writes compilation results and exits nonzero +if an expectation fails. It creates artifacts under the system temporary directory and does not +execute any compiled DEE. This verifies generation and compilation; it does not exercise deployment, +execution, debug mode or host-specific `alsoInclude` configuration. Normal unit tests remain +independent of Product packages and do not execute product assemblies. + +## Added deployment content + +Pass **both** lists when scanning a PR: + +```bash +cmf-codereview scan --repository-root . \ + --changed-files-list changed-files.txt \ + --added-files-list added-files.txt \ + --output findings.json +``` + +Both files contain one repository-relative (or absolute in-repository) path per line. The added +list must describe added **or renamed** destinations, not all modified files. An empty list is valid. +Without the added list, deployment version validation is skipped; the reviewer warns when a PR +changed-file list was provided without it. It never guesses that historical edits are new files. + +`DeploymentVersionMismatch` (High) compares the new file's folder version with the owning Data +package's `cmfpackage.json` version from the checked-out PR head. Content is classified from +`contentToPack`: `MasterData`, `EntityTypes`, `ProcessRulesPre`, and `ProcessRulesPost`. This supports +Before/After and EntityTypes/EntityTypesAttributes folders. Unversioned content such as Services +and untouched historical folders are excluded. An unreadable manifest or missing version produces +`DeploymentVersionUnresolved` (Medium). + +For a package version `2.61.0`, a new `MasterData/2.60.0/data.json` or +`DEEs/ProcessRules/2.60.0/After/Rule.cs` is flagged with the expected destination under `2.61.0`. +The metadata finding is placed at line 1 of the new file. This repository-level check uses the +explicit added-file list, independently of the C# `--roots` selection, so MasterData-only PRs work. + +All new diagnostics participate in rule enable/severity configuration, suppression/baselines where +applicable, `--fail-on`, and the existing JSON, HTML, Markdown and SARIF outputs. + +## Changes to the Bash PR-Changes task + +Apply these edits to the consuming project's Bash PR-Changes task +once `CmfCodeReviewVersion` selects a published version containing this feature. Keep findings in +JSON for the existing inline-comment consumer. Restore/build must precede reference validation; +the reviewer does not perform restores or acquire feed credentials. + +**1. Add a path and generate the new-file list beside the existing changed-file list:** + +```bash +addedFilesPath="$(Agent.TempDirectory)/codereview-added-files.txt" +git -c core.quotepath=off diff --name-only --diff-filter=AR --find-renames \ + "origin/${targetBranch}...HEAD" > "$addedFilesPath" +``` + +Use the same merge-base range for both lists. Rename detection ensures that version-folder moves +are checked using their destination path. `--added-files-list` is a separate input so older release +files modified by the PR are not incorrectly treated as new content. + +**2. Add to `scanArgs` before invoking npx:** + +```bash +scanArgs+=(--added-files-list "$addedFilesPath") +``` + +**3. After the changed-hunk loop, add a line-1 anchor for added/renamed files:** + +```bash +while IFS= read -r file || [[ -n "$file" ]]; do + if [[ -n "$file" && -s "$(Build.SourcesDirectory)/$file" ]]; then + printf '%s\t1\t1\n' "$file" >> "$changedLinesPath" + fi +done < "$addedFilesPath" +``` + +This keeps the TSV format of the supplied Bash task. A pure rename can have no changed-line hunks; +without this anchor, a folder-version finding can fail the gate yet be omitted by the PR comment +step's changed-line filter. Existing overlapping ranges for added files are harmless. + +**4. Fix the findings counter's zero-result case under `set -euo pipefail`:** + +```bash +findingsCount="$(grep -c '^[[:space:]]*"RelativePath"[[:space:]]*:' "$findingsPath" || true)" +echo "Findings count: $findingsCount" +``` + +The original `grep -o ... | wc -l` returns nonzero when no finding matches. That can fail an otherwise +successful clean scan before output variables are exposed. The revised counter tolerates zero +findings while preserving the reviewer's exit code and JSON output. diff --git a/cmf-codereview/docs/rules.md b/cmf-codereview/docs/rules.md index dee86ba..cd17832 100644 --- a/cmf-codereview/docs/rules.md +++ b/cmf-codereview/docs/rules.md @@ -101,9 +101,10 @@ Guards recognize `HasProperty`, `TryGetEntityTypeProperty`, `ObjectExists`, `Try negation/comparisons, applicable branches, and early returns. Checks on another property or target do not protect an operation. Reachable local functions are analyzed, including guards at their call sites; dormant local functions and callbacks are excluded. Raw metadata changes recognize -`GenerateSchema` and `GenerateEntityTypeDBSchema` after the update. The normal -`IMicronEntityTypeService.UpdateEntityTypeProperties` route does not require a duplicate schema -call. A guarded removal followed by recreation supports deliberate attribute-definition repairs; +`GenerateSchema` and `GenerateEntityTypeDBSchema` after the update. The supported +`UpdateEntityTypeProperties` helper pattern is treated as schema-aware and does not require a +duplicate schema call; verify that the project's helper performs regeneration. A guarded removal +followed by recreation supports deliberate attribute-definition repairs; the removal still needs value cleanup. Schema context is read once per scan from the same Data package's versioned MasterData. It tracks @@ -195,3 +196,9 @@ carry the reverse link (`CalledByDee`). Findings flagged via `LoopInducedByCalle same reason) carry a `LoopExposure` chain describing the call path back to the loop that causes the repetition — rendered as prose by `CmLoopExposureFinder.FormatChain` and included in both the Standard and Detailed reports. + +## DEE references and deployment versions + +The reference checks and PR deployment-folder diagnostics are described in +[pr-validation.md](pr-validation.md), including their default severities, required metadata, +historical-folder exclusions, and CLI/pipeline inputs. diff --git a/cmf-codereview/src/Analysis/DeeReferenceRepository.cs b/cmf-codereview/src/Analysis/DeeReferenceRepository.cs new file mode 100644 index 0000000..bc3ec32 --- /dev/null +++ b/cmf-codereview/src/Analysis/DeeReferenceRepository.cs @@ -0,0 +1,241 @@ +using System.Collections.Concurrent; +using System.Reflection; +using System.Reflection.Emit; +using System.Reflection.Metadata; +using System.Reflection.Metadata.Ecma335; +using System.Reflection.PortableExecutable; +using System.Runtime.InteropServices; +using System.Text.Json; +using System.Xml.Linq; +using Cmf.Cli.Plugin.CodeReview.Rules; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; + +namespace Cmf.Cli.Plugin.CodeReview; + +/// Project-scoped compiler metadata. Referenced assemblies are read, never executed. +public sealed class DeeReferenceRepository(string repositoryRoot, CodeReviewFileCache cache) +{ + private readonly ConcurrentDictionary> _projects = new(StringComparer.OrdinalIgnoreCase); + + public DeeReferenceProject? ForFile(string file) + { + var root = Path.GetFullPath(repositoryRoot); + for (var directory = new DirectoryInfo(Path.GetDirectoryName(file)!); directory is not null; directory = directory.Parent) + { + var projects = directory.GetFiles("*.csproj"); + // Do not guess which of multiple projects owns this file. + if (projects.Length > 1) return null; + if (projects.Length == 1) + return _projects.GetOrAdd(projects[0].FullName, p => new Lazy(() => Build(p))).Value; + if (directory.FullName.Equals(root, StringComparison.OrdinalIgnoreCase)) break; + } + return null; + } + + private DeeReferenceProject Build(string project) + { + var paths = Directory.GetFiles(RuntimeEnvironment.GetRuntimeDirectory(), "*.dll") + .ToDictionary(p => Path.GetFileName(p)!, p => p, StringComparer.OrdinalIgnoreCase); + // The DEE compiler takes framework assemblies from the host runtime. NuGet's older + // System.* reference assets must not replace them and create conflicting identities. + var runtimeFiles = paths.Keys.ToHashSet(StringComparer.OrdinalIgnoreCase); + var trees = new Dictionary(StringComparer.OrdinalIgnoreCase); + var sourceAssemblies = new Dictionary(); + var sourceNamespaces = new Dictionary>(StringComparer.OrdinalIgnoreCase); + var visited = new HashSet(StringComparer.OrdinalIgnoreCase); + var metadataAvailable = false; + var complete = true; + void AddReference(string path) + { + if (!runtimeFiles.Contains(Path.GetFileName(path))) paths[Path.GetFileName(path)] = path; + metadataAvailable = true; + } + void Visit(string path) + { + path = Path.GetFullPath(path); + if (!visited.Add(path)) return; + var directory = Path.GetDirectoryName(path)!; + try + { + var xml = XDocument.Load(path); + var assemblyName = xml.Descendants().FirstOrDefault(e => e.Name.LocalName == "AssemblyName")?.Value ?? Path.GetFileNameWithoutExtension(path); + var namespaces = new HashSet(StringComparer.Ordinal); + sourceNamespaces[assemblyName] = namespaces; + foreach (var file in SourceFiles(directory)) + { + var syntax = cache.Get(file); + trees.TryAdd(file, syntax.Root.SyntaxTree); + sourceAssemblies.TryAdd(syntax.Root.SyntaxTree, assemblyName); + foreach (var ns in syntax.Root.DescendantNodes().OfType()) + namespaces.Add(string.Join(".", ns.Ancestors().OfType().Reverse().Select(n => n.Name.ToString()).Append(ns.Name.ToString()))); + } + var assets = Path.Combine(directory, "obj", "project.assets.json"); + if (File.Exists(assets)) + { + using var document = JsonDocument.Parse(File.ReadAllText(assets)); + var json = document.RootElement; + if (json.TryGetProperty("targets", out var targets) && json.TryGetProperty("libraries", out var libraries) && + json.TryGetProperty("packageFolders", out var folders)) + { + var target = targets.EnumerateObject().FirstOrDefault(t => !t.Name.Contains('/')); + if (target.Value.ValueKind == JsonValueKind.Undefined) target = targets.EnumerateObject().First(); + foreach (var library in target.Value.EnumerateObject()) + { + if (!libraries.TryGetProperty(library.Name, out var info) || !info.TryGetProperty("path", out var relative) || + !info.TryGetProperty("type", out var type) || type.GetString() != "package") continue; + var group = library.Value.TryGetProperty("compile", out var compile) ? compile : + library.Value.TryGetProperty("runtime", out var runtime) ? runtime : default; + if (group.ValueKind != JsonValueKind.Object) continue; + foreach (var asset in group.EnumerateObject().Where(a => a.Name.EndsWith(".dll", StringComparison.OrdinalIgnoreCase))) + { + var found = folders.EnumerateObject().Select(f => Path.Combine(f.Name, + Local(relative.GetString()!), Local(asset.Name))).FirstOrDefault(File.Exists); + if (found is null) { complete = false; continue; } + AddReference(found); + } + } + } + } + foreach (var reference in xml.Descendants().Where(e => e.Name.LocalName == "Reference")) + { + var hint = reference.Elements().FirstOrDefault(e => e.Name.LocalName == "HintPath")?.Value; + if (hint is null) continue; + var full = Path.GetFullPath(Path.Combine(directory, Local(hint))); + if (File.Exists(full)) AddReference(full); + else complete = false; + } + foreach (var reference in xml.Descendants().Where(e => e.Name.LocalName == "ProjectReference")) + if (reference.Attribute("Include")?.Value is { } include) Visit(Path.Combine(directory, Local(include))); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or JsonException or System.Xml.XmlException or InvalidOperationException) + { complete = false; } + } + Visit(project); + var references = new List(); + var namespaces = new Dictionary>(StringComparer.OrdinalIgnoreCase); + HashSet defaultNamespaces = new(StringComparer.Ordinal), defaultAssemblies = new(StringComparer.OrdinalIgnoreCase); + foreach (var assembly in paths.Values) + { + try + { + using var stream = File.OpenRead(assembly); + using var pe = new PEReader(stream); + if (!pe.HasMetadata) continue; + var reader = pe.GetMetadataReader(); + var name = reader.GetString(reader.GetAssemblyDefinition().Name); + var names = new HashSet(StringComparer.Ordinal); + foreach (var handle in reader.TypeDefinitions) + { + var type = reader.GetTypeDefinition(handle); + var visibility = type.Attributes & TypeAttributes.VisibilityMask; + if (visibility == TypeAttributes.Public && !type.Namespace.IsNil) names.Add(reader.GetString(type.Namespace)); + } + foreach (var handle in reader.ExportedTypes) + { + var type = reader.GetExportedType(handle); + if (!type.Namespace.IsNil) names.Add(reader.GetString(type.Namespace)); + } + namespaces[name] = names; + if (name == "Cmf.Foundation.Common") ReadDefaults(pe, reader, defaultNamespaces, defaultAssemblies); + references.Add(MetadataReference.CreateFromFile(assembly)); + } + catch (Exception ex) when (ex is IOException or BadImageFormatException or UnauthorizedAccessException) { complete = false; } + } + foreach (var source in sourceNamespaces) + namespaces.TryAdd(source.Key, source.Value); + var compilation = CSharpCompilation.Create("DeeReferenceAnalysis", trees.Values, references, + new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + return new(compilation, namespaces, defaultNamespaces, defaultAssemblies, metadataAvailable, complete, sourceAssemblies, sourceNamespaces.Keys.ToHashSet(StringComparer.OrdinalIgnoreCase)); + } + + private static string Local(string path) => path.Replace('\\', Path.DirectorySeparatorChar).Replace('/', Path.DirectorySeparatorChar); + private static readonly HashSet Noise = new(StringComparer.OrdinalIgnoreCase) + { "bin", "obj", ".git", "node_modules", "packages", "TestResults" }; + private static IEnumerable SourceFiles(string directory) + { + foreach (var file in Directory.EnumerateFiles(directory, "*.cs")) yield return Path.GetFullPath(file); + foreach (var child in Directory.EnumerateDirectories(directory).Where(d => !Noise.Contains(Path.GetFileName(d)))) + foreach (var file in SourceFiles(child)) yield return file; + } + + private static readonly Dictionary Opcodes = typeof(OpCodes).GetFields(BindingFlags.Public | BindingFlags.Static) + .Where(f => f.FieldType == typeof(OpCode)).Select(f => (OpCode)f.GetValue(null)!).ToDictionary(o => o.Value); + + // These arrays are initialized as literals by the platform compiler. Decode only IL operands; + // never invoke the type initializer or any product method. Unknown layouts stay unresolved. + private static void ReadDefaults(PEReader pe, MetadataReader reader, HashSet usings, HashSet assemblies) + { + foreach (var handle in reader.TypeDefinitions) + { + var type = reader.GetTypeDefinition(handle); + if (reader.GetString(type.Name) != "Compiler" || + reader.GetString(type.Namespace) != "Cmf.Foundation.Common.DynamicExecutionEngine") continue; + foreach (var methodHandle in type.GetMethods()) + { + var method = reader.GetMethodDefinition(methodHandle); + if (reader.GetString(method.Name) != ".cctor" || method.RelativeVirtualAddress == 0) continue; + var bytes = pe.GetMethodBody(method.RelativeVirtualAddress).GetILBytes()!; + var pending = new List(); + var foundFields = new HashSet(); + var decodedNamespaces = new HashSet(StringComparer.Ordinal); + var decodedAssemblies = new HashSet(StringComparer.OrdinalIgnoreCase); + var unsupportedInitializer = false; + var initializerHasCall = false; + for (var i = 0; i < bytes.Length;) + { + short code = bytes[i++]; + if (code == 0xfe) code = (short)(0xfe00 | bytes[i++]); + if (!Opcodes.TryGetValue(code, out var opcode)) return; + var length = opcode.OperandType switch + { + OperandType.InlineNone => 0, + OperandType.ShortInlineBrTarget or OperandType.ShortInlineI or OperandType.ShortInlineVar => 1, + OperandType.InlineVar => 2, + OperandType.InlineI8 or OperandType.InlineR => 8, + OperandType.InlineSwitch => 4 + 4 * BitConverter.ToInt32(bytes, i), + _ => 4 + }; + if (opcode.FlowControl == FlowControl.Call) initializerHasCall = true; + if (opcode == OpCodes.Ldstr) pending.Add(reader.GetUserString(MetadataTokens.UserStringHandle(BitConverter.ToInt32(bytes, i) & 0xffffff))); + if (opcode == OpCodes.Stsfld) + { + var fieldHandle = MetadataTokens.Handle(BitConverter.ToInt32(bytes, i)); + if (fieldHandle.Kind == HandleKind.FieldDefinition) + { + var field = reader.GetString(reader.GetFieldDefinition((FieldDefinitionHandle)fieldHandle).Name); + if (field is "CoreNamespaces" or "MesNamespaces") + { + foundFields.Add(field); + unsupportedInitializer |= initializerHasCall; + decodedNamespaces.UnionWith(pending); + } + if (field is "RuntimeReferences" or "CoreReferences" or "MesReferences") + { + foundFields.Add(field); + unsupportedInitializer |= initializerHasCall; + decodedAssemblies.UnionWith(pending.Where(s => s.EndsWith(".dll", StringComparison.OrdinalIgnoreCase)).Select(s => Path.GetFileNameWithoutExtension(s))); + } + } + pending.Clear(); + initializerHasCall = false; + } + i += length; + } + if (!unsupportedInitializer && foundFields.SetEquals(new[] { "CoreNamespaces", "MesNamespaces", "RuntimeReferences", "CoreReferences", "MesReferences" })) + { + usings.UnionWith(decodedNamespaces); + assemblies.UnionWith(decodedAssemblies); + } + } + } + } +} + +public sealed record DeeReferenceProject(CSharpCompilation Compilation, + Dictionary> AssemblyNamespaces, HashSet DefaultNamespaces, + HashSet DefaultAssemblies, bool MetadataAvailable, bool Complete, + Dictionary SourceAssemblies, HashSet SourceAssemblyNames) +{ + public bool DefaultsAvailable => DefaultNamespaces.Count > 0 && DefaultAssemblies.Count > 0; +} \ No newline at end of file diff --git a/cmf-codereview/src/Analysis/DeploymentVersionValidator.cs b/cmf-codereview/src/Analysis/DeploymentVersionValidator.cs new file mode 100644 index 0000000..f6cc0dd --- /dev/null +++ b/cmf-codereview/src/Analysis/DeploymentVersionValidator.cs @@ -0,0 +1,111 @@ +using System.Text.Json; +using System.Text.RegularExpressions; +using Cmf.Cli.Plugin.CodeReview.Enums; +using Cmf.Cli.Plugin.CodeReview.Models; + +namespace Cmf.Cli.Plugin.CodeReview; + +/// Release-folder checks apply only to explicitly added/renamed files, never historical directories. +public static class DeploymentVersionValidator +{ + public static IEnumerable GetFindings(string repositoryRoot, IEnumerable addedFiles, + IEnumerable? changedFiles = null, CodeReviewSettings? settings = null, CodeReviewBaseline? baseline = null) + { + var root = Path.GetFullPath(repositoryRoot); + var changed = changedFiles?.Select(p => FullPath(root, p)).Where(p => p is not null).ToHashSet(StringComparer.OrdinalIgnoreCase); + foreach (var path in addedFiles.Select(p => FullPath(root, p)).Where(p => p is not null).Distinct(StringComparer.OrdinalIgnoreCase).OrderBy(p => p, StringComparer.Ordinal)) + { + if (!File.Exists(path) || changed is not null && !changed.Contains(path)) continue; + var manifest = FindManifest(root, path!); + if (manifest is null) continue; + var relative = Path.GetRelativePath(Path.GetDirectoryName(manifest)!, path!).Replace('\\', '/'); + // A malformed owning manifest is actionable only for deployment-content candidates. + var candidate = relative.StartsWith("MasterData/", StringComparison.OrdinalIgnoreCase) || + relative.StartsWith("DEEs/ProcessRules/", StringComparison.OrdinalIgnoreCase); + string? problem = null, fix = null; + var rule = "DeploymentVersionUnresolved"; + try + { + using var document = JsonDocument.Parse(File.ReadAllText(manifest), new JsonDocumentOptions { AllowTrailingCommas = true, CommentHandling = JsonCommentHandling.Skip }); + var json = document.RootElement; + if (!json.TryGetProperty("packageType", out var type) || !string.Equals(type.GetString(), "Data", StringComparison.OrdinalIgnoreCase)) continue; + if (!json.TryGetProperty("contentToPack", out var entries) || entries.ValueKind != JsonValueKind.Array) continue; + foreach (var entry in entries.EnumerateArray()) + { + if (!entry.TryGetProperty("contentType", out var contentType) || contentType.GetString() is not { } content || + !new[] { "MasterData", "EntityTypes", "ProcessRulesPre", "ProcessRulesPost" }.Contains(content, StringComparer.OrdinalIgnoreCase) || + !entry.TryGetProperty("source", out var sourceNode) || sourceNode.GetString() is not { } source) continue; + var normalized = source.Replace('\\', '/'); + if (normalized.StartsWith("./", StringComparison.Ordinal)) normalized = normalized[2..]; + // Match the version segment independently of its expected value, otherwise a wrong + // directory would disappear from the very contentToPack glob we are validating. + var segments = normalized.Split('/'); + var versionIndex = Array.FindIndex(segments, s => s == "$(version)" || Regex.IsMatch(s, @"^\d+\.\d+\.\d+(?:\.\d+)?(?:[-+][^/]+)?$")); + if (versionIndex < 0) continue; // Recurring Services and other unversioned content. + var pattern = ""; + for (var i = 0; i < segments.Length; i++) + { + if (segments[i] == "**" && i < segments.Length - 1) { pattern += "(?:[^/]+/)*"; continue; } + pattern += i == versionIndex ? "(?[^/]+)" : + Regex.Escape(segments[i]).Replace(@"\*", ".*").Replace(@"\?", "."); + if (i < segments.Length - 1) pattern += "/"; + } + var match = Regex.Match(relative, "^" + pattern + "$", RegexOptions.CultureInvariant | RegexOptions.IgnoreCase); + if (!match.Success) continue; + if (!json.TryGetProperty("version", out var versionNode) || versionNode.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(versionNode.GetString())) + { + problem = $"Cannot validate this new deployment file: owning manifest '{Path.GetRelativePath(root, manifest)}' has no usable package version."; + fix = "Set the Data package version in cmfpackage.json and rerun the review."; + } + else + { + var expected = versionNode.GetString()!; + var actual = match.Groups["version"]; + if (string.Equals(actual.Value, expected, StringComparison.Ordinal)) break; + var destination = relative[..actual.Index] + expected + relative[(actual.Index + actual.Length)..]; + rule = "DeploymentVersionMismatch"; + problem = $"New {content} file uses folder version '{actual.Value}', but owning manifest '{Path.GetRelativePath(root, manifest)}' declares '{expected}'."; + fix = $"Move this file to '{Path.GetRelativePath(root, Path.Combine(Path.GetDirectoryName(manifest)!, destination.Replace('/', Path.DirectorySeparatorChar)))}', or correct the package version if the release target is wrong."; + } + break; + } + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or JsonException or InvalidOperationException) + { + if (candidate) + { + problem = $"Cannot read owning package manifest '{Path.GetRelativePath(root, manifest)}' to validate the new deployment folder version: {ex.Message}"; + fix = "Correct/restore the package manifest and rerun the review."; + } + } + if (problem is null || settings?.IsRuleEnabled(rule) == false) continue; + var lines = File.ReadAllLines(path!); + var snippet = lines.FirstOrDefault()?.Trim() ?? ""; + var relativePath = Path.GetRelativePath(root, path!); + if (CodeReviewSuppressions.IsSuppressed(lines, 1, rule) || baseline?.Contains(relativePath, rule, snippet) == true) continue; + yield return new Finding + { + FilePath = path!, RelativePath = relativePath, LineNumber = 1, + Rule = rule, Severity = settings?.GetSeverityOverride(rule) ?? (rule == "DeploymentVersionMismatch" ? Severity.High : Severity.Medium), + Description = problem, SuggestedFix = fix!, Snippet = snippet + }; + } + } + + private static string? FullPath(string root, string path) + { + var full = Path.GetFullPath(Path.Combine(root, path.Replace('\\', Path.DirectorySeparatorChar).Replace('/', Path.DirectorySeparatorChar))); + var relative = Path.GetRelativePath(root, full); + return relative == ".." || relative.StartsWith(".." + Path.DirectorySeparatorChar, StringComparison.Ordinal) || Path.IsPathRooted(relative) ? null : full; + } + private static string? FindManifest(string root, string file) + { + for (var directory = new DirectoryInfo(Path.GetDirectoryName(file)!); directory is not null; directory = directory.Parent) + { + var candidate = Path.Combine(directory.FullName, "cmfpackage.json"); + if (File.Exists(candidate)) return candidate; + if (directory.FullName.Equals(root, StringComparison.OrdinalIgnoreCase)) break; + } + return null; + } +} \ No newline at end of file diff --git a/cmf-codereview/src/CmFrameworkScanner.cs b/cmf-codereview/src/CmFrameworkScanner.cs index 0953fe9..88d9df7 100644 --- a/cmf-codereview/src/CmFrameworkScanner.cs +++ b/cmf-codereview/src/CmFrameworkScanner.cs @@ -225,13 +225,16 @@ public static bool AnyAtOrAboveSeverity(IEnumerable findings, Severity /// .codereview.baseline.json (when present and is true) are excluded. /// public static List GetFindings(string repositoryRoot, IEnumerable? roots = null, IEnumerable? changedFiles = null, - CodeReviewSettings? settings = null, bool applyBaseline = true, EntityCatalog? entityCatalog = null) + CodeReviewSettings? settings = null, bool applyBaseline = true, EntityCatalog? entityCatalog = null, IEnumerable? addedFiles = null) { roots ??= CodeReviewPaths.ResolveRoots(repositoryRoot, ReviewScope.Both); settings ??= CodeReviewSettings.Load(repositoryRoot); var baseline = applyBaseline ? CodeReviewBaseline.Load(repositoryRoot) : null; + var fileCache = new CodeReviewFileCache(); + changedFiles = changedFiles?.ToArray(); var customRules = settings?.BuildCustomRules(); - var activeRules = customRules is { Count: > 0 } ? Rules.Concat(customRules).ToList() : Rules; + var builtInRules = Rules.Concat(new IReviewRule[] { new DeeReferenceRule(new DeeReferenceRepository(repositoryRoot, fileCache)) }).ToList(); + var activeRules = customRules is { Count: > 0 } ? builtInRules.Concat(customRules).ToList() : builtInRules; var files = CodeReviewPaths.GetFiles(repositoryRoot, roots); // Resolve the entity catalog from the repo's referenced assemblies unless a caller injected one @@ -267,7 +270,6 @@ void Mark(string phase) } } - var fileCache = new CodeReviewFileCache(); // Warm the cache in parallel across every file the pre-passes and rule loop will touch: // parsing dominates scan time and each file's artifacts are independent, so this is where // multicore actually pays. The passes below then run over already-parsed trees. @@ -327,6 +329,8 @@ void Mark(string phase) serviceEndpointMap, deeClassNameByFile, deeInvocationMap, orchestrationCallSitesInDees, activeRules, repositoryRoot, entityCatalog, processRuleRepository)) .ToList(); Mark("rule loop"); + if (addedFiles is not null) + findings.AddRange(DeploymentVersionValidator.GetFindings(repositoryRoot, addedFiles, changedFiles, settings, baseline)); return findings .OrderBy(f => f.Severity switch { Severity.Critical => 0, Severity.High => 1, _ => 2 }) diff --git a/cmf-codereview/src/CodeReviewPaths.cs b/cmf-codereview/src/CodeReviewPaths.cs index a3a7a05..941f815 100644 --- a/cmf-codereview/src/CodeReviewPaths.cs +++ b/cmf-codereview/src/CodeReviewPaths.cs @@ -35,7 +35,7 @@ private record PackageManifest(string Directory, string? PackageType, List<(stri // File-classification regexes are structural (any "Cmf..Business/Cmf..Orchestration/" // shaped segment matches), not tied to the literal "Custom" project name -- so a customization // project using any other name (e.g. "Cmf.Semi.*") is classified correctly. Both the nested - // Micron-style DEEs path (Cmf..Data/Cmf..Data.Packages/DEEs) and the flatter convention + // nested DEEs path (Cmf..Data/Cmf..Data.Packages/DEEs) and the flatter convention // (Cmf..Data/DEEs) are matched. Kept independent of manifest-based root discovery -- these // classify individual files by shape, not by resolving package metadata. private static readonly Regex OrchestrationSegmentRegex = @@ -125,7 +125,9 @@ private static ResolvedRoots DiscoverUncached(string fullRepositoryRoot) !other.Equals(root, StringComparison.OrdinalIgnoreCase) && root.StartsWith(other + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase))).ToArray(); - if (orchestrationRoots.Length == 0 && deeRoots.Length == 0) + // MasterData-only Data packages still need added-file version validation without C# roots. + if (orchestrationRoots.Length == 0 && deeRoots.Length == 0 && + !manifests.Any(m => string.Equals(m.PackageType, "Data", StringComparison.OrdinalIgnoreCase))) { throw new CliException( $"No cmfpackage.json under '{fullRepositoryRoot}' resolves to a Business or Data package " + diff --git a/cmf-codereview/src/Commands/ScanCommand.cs b/cmf-codereview/src/Commands/ScanCommand.cs index 97d1563..89eddd4 100644 --- a/cmf-codereview/src/Commands/ScanCommand.cs +++ b/cmf-codereview/src/Commands/ScanCommand.cs @@ -25,6 +25,8 @@ public override void Configure(Command cmd) var changedFilesListOption = new Option("--changed-files-list", description: "Path to a text file listing changed files (one relative or absolute path per line, e.g. from 'git diff --name-only'). " + "When given, only findings on those files are reported, though the whole repository is still analyzed for cross-reference/loop-exposure context."); + var addedFilesListOption = new Option("--added-files-list", description: + "Added or renamed files in this PR, one path per line (git diff --name-only --diff-filter=AR -M). Required to validate deployment folder versions without flagging historical files."); var failOnOption = new Option("--fail-on", description: "If given, exit with a non-zero code when any reported finding is at or above this severity. If omitted, the command always exits 0 regardless of findings."); var updateBaselineOption = new Option("--update-baseline", description: @@ -48,21 +50,25 @@ public override void Configure(Command cmd) cmd.AddOption(rootsOption); cmd.AddOption(outputOption); cmd.AddOption(changedFilesListOption); + cmd.AddOption(addedFilesListOption); cmd.AddOption(failOnOption); cmd.AddOption(updateBaselineOption); cmd.AddOption(reportFormatOption); - cmd.Handler = CommandHandler.Create(Execute); + cmd.Handler = CommandHandler.Create(Execute); } - public int Execute(string repositoryRoot, ReviewScope scope, string[] roots, string? output, string? changedFilesList, Severity? failOn, bool updateBaseline, string? reportFormat = null) + public int Execute(string repositoryRoot, ReviewScope scope, string[] roots, string? output, string? changedFilesList, Severity? failOn, bool updateBaseline, string? reportFormat = null, string? addedFilesList = null) { var effectiveRoots = roots is { Length: > 0 } ? roots : CodeReviewPaths.ResolveRoots(repositoryRoot, scope); var changedFiles = string.IsNullOrWhiteSpace(changedFilesList) ? null : File.ReadAllLines(changedFilesList).Select(line => line.Trim()).Where(line => line.Length > 0); - var findings = CmFrameworkScanner.GetFindings(repositoryRoot, effectiveRoots, changedFiles, applyBaseline: !updateBaseline); + var addedFiles = string.IsNullOrWhiteSpace(addedFilesList) ? null : File.ReadAllLines(addedFilesList).Select(line => line.Trim()).Where(line => line.Length > 0); + if (changedFilesList is not null && addedFiles is null) + Console.Error.WriteLine("warning: deployment folder-version validation is skipped without --added-files-list; changed files alone cannot distinguish new content from historical edits."); + var findings = CmFrameworkScanner.GetFindings(repositoryRoot, effectiveRoots, changedFiles, applyBaseline: !updateBaseline, addedFiles: addedFiles); if (updateBaseline) { diff --git a/cmf-codereview/src/Rules/DeeReferenceRule.cs b/cmf-codereview/src/Rules/DeeReferenceRule.cs new file mode 100644 index 0000000..15fcf7f --- /dev/null +++ b/cmf-codereview/src/Rules/DeeReferenceRule.cs @@ -0,0 +1,139 @@ +using Cmf.Cli.Plugin.CodeReview.Enums; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.CSharp.Syntax; + +namespace Cmf.Cli.Plugin.CodeReview.Rules; + +/// Checks reference declarations and type/extension imports used in deployed DEE methods. +public sealed class DeeReferenceRule(DeeReferenceRepository repository) : IReviewRule +{ + public IEnumerable EscalatableRuleNames => []; + + public IEnumerable FindMatches(RuleFileContext context) + { + foreach (var type in context.Root.DescendantNodes().OfType().Where(c => + c.BaseList?.Types.Any(t => CodeReviewSyntaxTree.LastIdentifierSegment(t.Type.ToString()) == "DeeDevBase") == true)) + { + var methods = type.Members.OfType().Where(m => + m.Identifier.ValueText is "DeeActionCode" or "DeeTestCondition" && m.Modifiers.Any(SyntaxKind.OverrideKeyword)).ToList(); + var calls = methods.SelectMany(m => m.DescendantNodes().OfType()) + .Where(c => c.Expression is IdentifierNameSyntax { Identifier.ValueText: "UseReference" } || + c.Expression is MemberAccessExpressionSyntax { Expression: ThisExpressionSyntax, Name.Identifier.ValueText: "UseReference" }).ToList(); + var project = repository.ForFile(context.FilePath); + var imports = project is { DefaultsAvailable: true } ? new HashSet(project.DefaultNamespaces, StringComparer.Ordinal) : new(StringComparer.Ordinal); + var assemblies = project is { DefaultsAvailable: true } ? new HashSet(project.DefaultAssemblies, StringComparer.OrdinalIgnoreCase) : new(StringComparer.OrdinalIgnoreCase); + foreach (var call in calls) + if (call.ArgumentList.Arguments.FirstOrDefault()?.Expression is LiteralExpressionSyntax literal && literal.IsKind(SyntaxKind.StringLiteralExpression) && literal.Token.ValueText.Length > 0) + assemblies.Add(AssemblyName(literal.Token.ValueText)); + var unresolvedAssemblies = new HashSet(StringComparer.OrdinalIgnoreCase); + bool dynamicReference = false; + foreach (var call in calls) + { + var args = call.ArgumentList.Arguments; + if (args.Count != 2 || args.Any(a => a.Expression is not LiteralExpressionSyntax literal || !literal.IsKind(SyntaxKind.StringLiteralExpression))) + { + dynamicReference = true; + yield return Match("DeeReferenceUnresolved", call, Severity.Medium, + "UseReference arguments are not two literal strings; the deployed reference cannot be verified statically.", + "Use a supported literal assembly/namespace declaration, or review how the DEE compiler extracts this reference."); + continue; + } + var assembly = ((LiteralExpressionSyntax)args[0].Expression).Token.ValueText; + var ns = ((LiteralExpressionSyntax)args[1].Expression).Token.ValueText; + // Both arguments are independent compiler inputs. An empty call is a no-op. + if (assembly.Length == 0 && ns.Length == 0) continue; + var name = AssemblyName(assembly); + if (ns.Length > 0) imports.Add(ns); + if (name.Length > 0) assemblies.Add(name); + if (project is null || !project.MetadataAvailable) continue; + if (name.Length > 0) + { + if (!project.AssemblyNamespaces.ContainsKey(name) && unresolvedAssemblies.Add(name)) + yield return Match("DeeReferenceUnresolved", call, Severity.Medium, + $"Referenced assembly '{assembly}' is not resolvable from this DEE project's restored dependencies or HintPaths.", + "Restore the project and check the assembly name, deployment path, and reference metadata."); + + } + if (project.DefaultsAvailable && ns.Length > 0 && + !assemblies.Any(a => project.AssemblyNamespaces.TryGetValue(a, out var known) && known.Contains(ns))) + { + var owners = project.AssemblyNamespaces.Where(a => a.Value.Contains(ns)).Select(a => a.Key).ToList(); + var provenMissing = owners.Count > 0 && !project.SourceAssemblyNames.Overlaps(assemblies) && assemblies.All(project.AssemblyNamespaces.ContainsKey); + yield return Match(provenMissing ? "DeeInvalidReference" : "DeeReferenceUnresolved", call, + provenMissing ? Severity.High : Severity.Medium, + $"Namespace '{ns}' is not verified in any declared or default assembly.", + owners.Count > 0 ? $"Declare its assembly explicitly, for example UseReference(\"{owners[0]}.dll\", \"{ns}\")." : "Check the namespace and restore the assembly that supplies it."); + } + } + if (project is null || !project.MetadataAvailable || !project.DefaultsAvailable) + { + yield return Match("DeeReferenceUnresolved", type.Identifier.Parent!, Severity.Medium, + "DEE reference completeness could not be checked: the owning project, restored dependencies, or compiler default imports are unavailable.", + "Restore the DEE project before scanning and ensure the referenced Cmf.Foundation.Common compiler metadata is available."); + continue; + } + if (!project.Complete) + yield return Match("DeeReferenceUnresolved", type, Severity.Medium, + "Some project references could not be resolved; missing-reference validation is limited to symbols that can be bound.", + "Restore/build referenced projects and check HintPaths before treating the reference check as complete."); + if (dynamicReference) continue; // A dynamic declaration might provide the apparently missing import. + var model = project.Compilation.GetSemanticModel(context.Root.SyntaxTree); + if (methods.SelectMany(m => model.GetDiagnostics(m.Span)).Any(d => d.Severity == DiagnosticSeverity.Error && d.Id is "CS0246" or "CS0234" or "CS0433" or "CS0104" or "CS0121")) + yield return Match("DeeReferenceUnresolved", type, Severity.Medium, + "Some types/namespaces in deployed DEE code could not be bound against the project metadata; their required references cannot be verified.", + "Restore/build the owning project and resolve missing types or namespaces before treating reference validation as complete."); + var reported = new HashSet<(string, string)>(); + foreach (var node in methods.SelectMany(m => m.DescendantNodes().OfType())) + { + var symbol = model.GetSymbolInfo(node).Symbol; + var usedType = symbol as INamedTypeSymbol; + var extension = symbol as IMethodSymbol; + if (usedType?.TypeKind == TypeKind.Error || usedType?.IsAnonymousType == true || usedType?.ContainingNamespace?.IsGlobalNamespace == true) continue; + var ns = usedType?.ContainingNamespace?.ToDisplayString() ?? + (extension?.ReducedFrom is not null ? extension.ContainingNamespace.ToDisplayString() : null); + var assembly = usedType?.ContainingAssembly?.Name ?? + (extension?.ReducedFrom is not null ? extension.ContainingAssembly.Name : null); + var containingType = usedType ?? extension?.ContainingType; + var declarationTree = containingType?.DeclaringSyntaxReferences.FirstOrDefault()?.SyntaxTree; + if (declarationTree is not null && project.SourceAssemblies.TryGetValue(declarationTree, out var sourceAssembly)) assembly = sourceAssembly; + if (ns is null || assembly is null || ns.Length == 0) continue; + if (!project.AssemblyNamespaces.ContainsKey(assembly)) continue; // Source-only project types are not a proven deployable reference. + var qualified = node.Parent is QualifiedNameSyntax || node.Parent is AliasQualifiedNameSyntax || + node.Parent is MemberAccessExpressionSyntax access && access.Name == node && model.GetSymbolInfo(access.Expression).Symbol is INamespaceSymbol; + var missingNamespace = !qualified && !imports.Contains(ns); + var missingAssembly = !assemblies.Contains(assembly); + if (missingAssembly && containingType is not null) + { + var metadataName = MetadataName(containingType); + missingAssembly = !project.Compilation.SourceModule.ReferencedAssemblySymbols.Where(a => assemblies.Contains(a.Name)) + .Any(a => ProvidesType(a, metadataName, containingType)); + } + if ((!missingNamespace && !missingAssembly) || !reported.Add((ns, assembly))) continue; + if (node is IdentifierNameSyntax identifier && model.GetAliasInfo(identifier) is not null) continue; + yield return Match("DeeMissingReference", node, Severity.High, + $"Deployed DEE code uses '{ns}' from '{assembly}', but its required {(missingNamespace ? "namespace import" : "assembly reference")} is not supplied by UseReference or compiler defaults.", + $"Add UseReference(\"{(missingAssembly ? assembly + ".dll" : "")}\", \"{(missingNamespace ? ns : "")}\") inside DeeActionCode."); + } + } + } + + private static bool ProvidesType(IAssemblySymbol assembly, string metadataName, INamedTypeSymbol expected) + { + var supplied = assembly.GetTypeByMetadataName(metadataName) ?? assembly.ResolveForwardedType(metadataName); + return SymbolEqualityComparer.Default.Equals(supplied, expected.OriginalDefinition); + } + private static string MetadataName(INamedTypeSymbol type) => + type.ContainingType is not null ? MetadataName(type.ContainingType) + "+" + type.MetadataName : + (type.ContainingNamespace.IsGlobalNamespace ? "" : type.ContainingNamespace.ToDisplayString() + ".") + type.MetadataName; + + private static string AssemblyName(string value) + { + var normalized = value.Replace('\\', '/'); + var prefix = normalized.LastIndexOf('%'); + if (prefix >= 0) normalized = normalized[(prefix + 1)..]; + return Path.GetFileNameWithoutExtension(normalized); + } + private static RuleMatch Match(string rule, SyntaxNode node, Severity severity, string description, string fix) => + new(rule, node.GetLocation().GetLineSpan().StartLinePosition.Line + 1, severity, description, fix); +} \ No newline at end of file diff --git a/cmf-codereview/src/Rules/ProcessRuleValidationRule.cs b/cmf-codereview/src/Rules/ProcessRuleValidationRule.cs index 25db153..808d91a 100644 --- a/cmf-codereview/src/Rules/ProcessRuleValidationRule.cs +++ b/cmf-codereview/src/Rules/ProcessRuleValidationRule.cs @@ -177,7 +177,7 @@ private static RuleMatch MutationMatch(ProcessRuleSyntax syntax, InvocationExpre private static RuleMatch SchemaMatch(SyntaxNode node, string target) => Match("ProcessRuleSchemaRegeneration", node, Severity.High, $"Raw entity-type property update for '{target}' has no recognized subsequent schema regeneration on the same execution path.", - "Call GenerateEntityTypeDBSchema for the same entity type or its GenerateSchema method after the update. IMicronEntityTypeService.UpdateEntityTypeProperties already handles regeneration."); + "Call GenerateEntityTypeDBSchema for the same entity type or its GenerateSchema method after the update. For UpdateEntityTypeProperties, verify that the project's helper handles schema regeneration."); private static bool IsSchemaCall(InvocationExpressionSyntax call, string target) => Name(call) == "GenerateSchema" && Tokens(Receiver(call)) == target || diff --git a/cmf-codereview/src/Rules/RuleFileContext.cs b/cmf-codereview/src/Rules/RuleFileContext.cs index ec0f880..5bd6564 100644 --- a/cmf-codereview/src/Rules/RuleFileContext.cs +++ b/cmf-codereview/src/Rules/RuleFileContext.cs @@ -10,6 +10,7 @@ namespace Cmf.Cli.Plugin.CodeReview.Rules; public class RuleFileContext { public required CompilationUnitSyntax Root { get; init; } + public string FilePath { get; init; } = ""; public required IReadOnlyList Lines { get; init; } public required LoopState LoopState { get; init; } public required List MethodRanges { get; init; } @@ -85,6 +86,7 @@ private static RuleFileContext Build(CompilationUnitSyntax root, IReadOnlyList.Data/Cmf..Data.Packages/DEEs) under a + // The nested shape (Cmf..Data/Cmf..Data.Packages/DEEs) under a // non-"Custom" project name -- the DEE root comes from the manifest's own contentToPack source, // so both DEEs conventions resolve without any special-casing. var packagesDir = Path.Combine(_tempRoot, "Cmf.Foo.Data", "Cmf.Foo.Data.Packages"); diff --git a/cmf-codereview/tests/DeeReferenceTests.cs b/cmf-codereview/tests/DeeReferenceTests.cs new file mode 100644 index 0000000..86735a5 --- /dev/null +++ b/cmf-codereview/tests/DeeReferenceTests.cs @@ -0,0 +1,200 @@ +using System.Collections.Concurrent; +using System.Runtime.InteropServices; +using System.Reflection.PortableExecutable; +using Cmf.Cli.Plugin.CodeReview; +using Cmf.Cli.Plugin.CodeReview.Rules; +using Cmf.Cli.Plugin.CodeReview.Models; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Xunit; + +namespace codereview.tests; + +public sealed class DeeReferenceTests : IDisposable +{ + private readonly string _root = Path.Combine(Path.GetTempPath(), "dee-references-" + Guid.NewGuid()); + private readonly string _project; + private readonly string _file; + private readonly CodeReviewFileCache _cache = new(); + + public DeeReferenceTests() + { + _project = Path.Combine(_root, "Cmf.Test.Data", "DEEs"); + Directory.CreateDirectory(_project); + _file = Path.Combine(_project, "Example.cs"); + Emit("Cmf.Foundation.Common", """ + namespace Cmf.Foundation.Common + { + public class DeeDevBase + { + protected void UseReference(string assembly, string ns) { } + public virtual void DeeActionCode() { } + public virtual bool DeeTestCondition() => true; + } + } + namespace Cmf.Foundation.Common.DynamicExecutionEngine + { + public static class Compiler + { + public static readonly string[] CoreNamespaces = new[] { "System", "System.Collections.Generic", "Cmf.Foundation.Common" }; + public static readonly string[] MesNamespaces = new string[0]; + public static readonly string[] CoreReferences = new[] { "System.Private.CoreLib.dll", "System.Console.dll", "Cmf.Foundation.Common.dll" }; + public static readonly string[] MesReferences = new string[0]; + public static readonly string[] RuntimeReferences = new string[0]; + static Compiler() { throw new System.InvalidOperationException("This code must never execute during analysis"); } + } + } + """); + Emit("Custom.Library", """ + namespace Custom.Tools { public class Widget { } } + namespace Custom.Extensions { public static class Extensions { public static int Extra(this string value) => value.Length; } } + """); + File.WriteAllText(Path.Combine(_project, "Dees.csproj"), """ + net8.0 + + Cmf.Foundation.Common.dll + Custom.Library.dll + + """); + } + + private void Emit(string name, string source) + { + var refs = Directory.GetFiles(RuntimeEnvironment.GetRuntimeDirectory(), "*.dll").Where(p => { using var stream = File.OpenRead(p); using var pe = new PEReader(stream); return pe.HasMetadata; }).Select(p => MetadataReference.CreateFromFile(p)); + var compilation = CSharpCompilation.Create(name, [CSharpSyntaxTree.ParseText(source)], refs, + new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + var result = compilation.Emit(Path.Combine(_project, name + ".dll")); + Assert.True(result.Success, string.Join("; ", result.Diagnostics)); + } + + private List Review(string body, string imports = "using Custom.Tools;") + { + File.WriteAllText(_file, $$""" + using Cmf.Foundation.Common; + using System; + {{imports}} + public class Example : DeeDevBase + { + public override void DeeActionCode() + { + {{body}} + } + } + """); + var context = RuleFileContext.Build(_cache.Get(_file), new(), new ConcurrentDictionary(), EntityCatalog.Empty); + return new DeeReferenceRule(new DeeReferenceRepository(_root, _cache)).FindMatches(context).ToList(); + } + + [Fact] + public void ValidExplicitReference_ResolvesNamespaceAndAssembly() => + Assert.Empty(Review("UseReference(\"Custom.Library.dll\", \"Custom.Tools\"); var value = new Widget();")); + + [Fact] + public void UsedNamespaceMissingItsReference_IsFlaggedAtTheUse() => + Assert.Contains(Review("var value = new Widget();"), f => f.RuleName == "DeeMissingReference" && f.Severity == Cmf.Cli.Plugin.CodeReview.Enums.Severity.High); + + [Fact] + public void UnusedUsingDoesNotRequireAReference() => Assert.Empty(Review("Console.WriteLine(1);")); + + [Fact] + public void NamespaceAbsentFromAllSelectedAssemblies_IsInvalid() => + Assert.Contains(Review("UseReference(\"System.Private.CoreLib.dll\", \"Custom.Tools\");"), f => f.RuleName == "DeeInvalidReference"); + + [Fact] + public void NamespaceCanBeSuppliedByADifferentDefaultAssembly() => + Assert.Empty(Review("UseReference(\"Custom.Library.dll\", \"System\"); new Custom.Tools.Widget();", "")); + + [Fact] + public void NamespaceCanBeSuppliedByAnotherDeclaredAssembly() => + Assert.Empty(Review("UseReference(\"System.Console.dll\", \"Custom.Tools\"); UseReference(\"Custom.Library.dll\", \"\"); new Widget();")); + + [Theory] + [InlineData("System.Runtime", "namespace Legacy { public class Placeholder { } }")] + [InlineData("System.Collections", "namespace System.Collections.Generic { public class Dictionary { } }")] + public void LegacyPackageFrameworkReferencesDoNotReplaceRuntimeAssemblies(string assembly, string source) + { + Emit(assembly, "[assembly: System.Reflection.AssemblyVersion(\"4.0.0.0\")]\n" + source); + var project = Path.Combine(_project, "Dees.csproj"); + File.WriteAllText(project, File.ReadAllText(project).Replace("", + $"{assembly}.dll")); + Assert.Empty(Review("Console.WriteLine(new Dictionary().Count);", "using System.Collections.Generic;")); + } + + [Fact] + public void UnknownAssembly_IsUnresolvedRatherThanProvenInvalid() => + Assert.Contains(Review("UseReference(\"Unavailable.dll\", \"Custom.Tools\");"), f => f.RuleName == "DeeReferenceUnresolved"); + + [Fact] + public void EmptyAssemblyForFrameworkNamespace_IsValid() => + Assert.Empty(Review("UseReference(\"\", \"System.Threading\"); Thread.Sleep(0);", "using System.Threading;")); + + [Fact] + public void EmptyAssemblyForCustomNamespace_RequiresAnExplicitAssembly() => + Assert.Contains(Review("UseReference(\"\", \"Custom.Tools\"); new Widget();"), f => f.RuleName == "DeeInvalidReference"); + + [Fact] + public void AssemblyOnlyReferenceWithFullyQualifiedType_IsValid() => + Assert.Empty(Review("UseReference(\"Custom.Library.dll\", \"\"); new Custom.Tools.Widget();", "")); + + [Fact] + public void MissingReferenceForExtensionImport_IsFlagged() => + Assert.Contains(Review("var length = \"text\".Extra();", "using Custom.Extensions;"), f => f.RuleName == "DeeMissingReference"); + + [Fact] + public void DynamicDeclaration_IsUnresolvedAndDoesNotInventMissingReferences() + { + var findings = Review("var ns = \"Custom.Tools\"; UseReference(\"Custom.Library.dll\", ns); new Widget();"); + Assert.Contains(findings, f => f.RuleName == "DeeReferenceUnresolved"); + Assert.DoesNotContain(findings, f => f.RuleName == "DeeMissingReference"); + } + + [Fact] + public void RuntimeDefaultsAreReadWithoutExecutingProductCode() + { + Review("Console.WriteLine(1);"); + var project = new DeeReferenceRepository(_root, _cache).ForFile(_file)!; + Assert.True(project.DefaultsAvailable); + Assert.Contains("System", project.DefaultNamespaces); + Assert.Contains("System.Private.CoreLib", project.DefaultAssemblies); + } + + [Fact] + public void MissingOwningProject_ProducesValidationLimitation() + { + File.Delete(Path.Combine(_project, "Dees.csproj")); + Assert.Contains(Review("Console.WriteLine(1);"), f => f.RuleName == "DeeReferenceUnresolved"); + } + + [Fact] + public void EmptyDeclaration_IsANoOp() => Assert.Empty(Review("UseReference(\"\", \"\");")); + + [Fact] + public void EmptyAssemblyNamespaceCanUseAReferenceDeclaredLater() + { + Assert.Empty(Review("UseReference(\"\", \"Custom.Tools\"); UseReference(\"Custom.Library.dll\", \"\"); new Widget();")); + } + + [Fact] + public void ReferencedProjectNamespaceCanBeValidatedBeforeItsDllIsBuilt() + { + var common = Path.Combine(_root, "Common"); + Directory.CreateDirectory(common); + File.WriteAllText(Path.Combine(common, "Common.csproj"), """Project.Commonnet8.0"""); + File.WriteAllText(Path.Combine(common, "Widget.cs"), "namespace Project.Tools { public class SourceWidget { } }"); + var project = Path.Combine(_project, "Dees.csproj"); + File.WriteAllText(project, File.ReadAllText(project).Replace("", "")); + Assert.Empty(Review("UseReference(\"Project.Common.dll\", \"Project.Tools\"); new SourceWidget();", "using Project.Tools;")); + } + + [Fact] + public void UnboundTypeIsReportedAsAValidationLimitation() + { + Assert.Contains(Review("UnknownType value;"), f => f.RuleName == "DeeReferenceUnresolved"); + } + + [Fact] + public void AnonymousTypesDoNotInventGlobalNamespaceImports() => + Assert.Empty(Review("var item = new { Name = \"value\" }; Console.WriteLine(item.Name);")); + + public void Dispose() => Directory.Delete(_root, true); +} \ No newline at end of file diff --git a/cmf-codereview/tests/DeploymentVersionTests.cs b/cmf-codereview/tests/DeploymentVersionTests.cs new file mode 100644 index 0000000..21a10e2 --- /dev/null +++ b/cmf-codereview/tests/DeploymentVersionTests.cs @@ -0,0 +1,187 @@ +using System.CommandLine; +using System.Text.Json; +using Cmf.Cli.Plugin.CodeReview; +using Cmf.Cli.Plugin.CodeReview.Commands; +using Cmf.Cli.Plugin.CodeReview.Enums; +using Cmf.Cli.Plugin.CodeReview.Models; +using Cmf.Cli.Plugin.CodeReview.Rules; +using Xunit; + +namespace codereview.tests; + +public sealed class DeploymentVersionTests : IDisposable +{ + private readonly string _root = Path.Combine(Path.GetTempPath(), "deployment-versions-" + Guid.NewGuid()); + private string Package => Path.Combine(_root, "Cmf.Test.Data"); + private string Manifest => Path.Combine(Package, "cmfpackage.json"); + + public DeploymentVersionTests() + { + Directory.CreateDirectory(Package); + WriteManifest("2.61.0"); + } + private void WriteManifest(string version) => File.WriteAllText(Manifest, $$""" + { "packageType": "Data", "version": "{{version}}", "contentToPack": [ + { "contentType": "MasterData", "source": "MasterData/$(version)/*" }, + { "contentType": "EntityTypes", "source": "DEEs/ProcessRules/EntityTypes/$(version)/*" }, + { "contentType": "ProcessRulesPre", "source": "DEEs/ProcessRules/$(version)/Before/*" }, + { "contentType": "ProcessRulesPost", "source": "DEEs/ProcessRules/$(version)/After/*" }, + { "contentType": "ProcessRulesPost", "source": "DEEs/ProcessRules/EntityTypesAttributes/$(version)/*" }, + { "contentType": "ProcessRulesPost", "source": "DEEs/ProcessRules/Services/*" } + ] } + """); + private string Write(string relative) + { + var full = Path.Combine(Package, relative.Replace('/', Path.DirectorySeparatorChar)); + Directory.CreateDirectory(Path.GetDirectoryName(full)!); + File.WriteAllText(full, relative.EndsWith(".json") ? "{}" : "// deployment rule"); + return Path.GetRelativePath(_root, full); + } + private List Validate(IEnumerable added, IEnumerable? changed = null, CodeReviewSettings? settings = null, CodeReviewBaseline? baseline = null) => + DeploymentVersionValidator.GetFindings(_root, added, changed, settings, baseline).ToList(); + + [Theory] + [InlineData("MasterData/{0}/data.json")] + [InlineData("DEEs/ProcessRules/{0}/Before/Rule.cs")] + [InlineData("DEEs/ProcessRules/{0}/After/Rule.cs")] + [InlineData("DEEs/ProcessRules/EntityTypes/{0}/Rule.cs")] + [InlineData("DEEs/ProcessRules/EntityTypesAttributes/{0}/Rule.cs")] + public void EachVersionedContentType_ValidatesPackageVersion(string pattern) + { + var wrong = Write(string.Format(pattern, "2.60.0")); + var finding = Assert.Single(Validate([wrong])); + Assert.Equal("DeploymentVersionMismatch", finding.Rule); + Assert.Equal(Severity.High, finding.Severity); + Assert.Equal(1, finding.LineNumber); + Assert.Contains("2.61.0", finding.Description); + Assert.Contains("2.60.0", finding.Description); + Assert.Contains("cmfpackage.json", finding.Description); + Assert.Contains("2.61.0", finding.SuggestedFix); + var correct = Write(string.Format(pattern, "2.61.0")); + Assert.Empty(Validate([correct])); + } + + [Fact] + public void HistoricalEditIsNotAnAddedFile_AndServicesAreUnversioned() + { + var old = Write("MasterData/2.59.0/data.json"); + Assert.Empty(Validate([], [old])); + Assert.Empty(Validate([Write("DEEs/ProcessRules/Services/Rule.cs")])); + } + + [Fact] + public void RenamedDestinationAndWindowsPathsAreValidated() => + Assert.Single(Validate([Write("MasterData/2.60.0/renamed.json").Replace('/', '\\')])); + + [Fact] + public void VersionComesFromTheCurrentManifest_IncludingAVersionBump() + { + var file = Write("MasterData/2.61.0/data.json"); + Assert.Empty(Validate([file])); + WriteManifest("2.62.0"); + Assert.Contains("2.62.0", Assert.Single(Validate([file])).Description); + } + + [Fact] + public void NearestPackageOwnsItsVersion() + { + var nested = Path.Combine(Package, "Nested"); + Directory.CreateDirectory(nested); + File.WriteAllText(Path.Combine(nested, "cmfpackage.json"), """{"packageType":"Data","version":"9.0.0","contentToPack":[{"contentType":"MasterData","source":"MasterData/$(version)/*"}]}"""); + Assert.Empty(Validate([Write("Nested/MasterData/9.0.0/data.json")])); + } + + [Fact] + public void MissingVersionAndMalformedManifestAreUnresolved() + { + var file = Write("MasterData/2.60.0/data.json"); + File.WriteAllText(Manifest, File.ReadAllText(Manifest).Replace("\"version\": \"2.61.0\",", "")); + Assert.Equal("DeploymentVersionUnresolved", Assert.Single(Validate([file])).Rule); + File.WriteAllText(Manifest, "{"); + Assert.Equal("DeploymentVersionUnresolved", Assert.Single(Validate([file])).Rule); + } + + [Fact] + public void ChangedFileFilterDeletedFilesAndFilesOutsideRootAreRespected() + { + var file = Write("MasterData/2.60.0/data.json"); + Assert.Empty(Validate([file], [])); + Assert.Empty(Validate([Path.Combine(_root, "..", "outside.json")])); + File.Delete(Path.Combine(_root, file)); + Assert.Empty(Validate([file])); + } + + [Fact] + public void SettingsAndBaselineApplyToMetadataFindings() + { + var file = Write("MasterData/2.60.0/data.json"); + var settings = new CodeReviewSettings { Rules = new() { ["DeploymentVersionMismatch"] = new RuleSetting { Enabled = false } } }; + Assert.Empty(Validate([file], settings: settings)); + settings.Rules["DeploymentVersionMismatch"] = new() { Severity = "Critical" }; + Assert.Equal(Severity.Critical, Assert.Single(Validate([file], settings: settings)).Severity); + CodeReviewBaseline.Write(_root, Validate([file])); + Assert.Empty(Validate([file], baseline: CodeReviewBaseline.Load(_root))); + } + + [Fact] + public async Task PipelineStyleInvocation_ReportsNewMasterDataJsonAndFailsTheGate() + { + var file = Write("MasterData/2.60.0/data.json"); + var list = Path.Combine(_root, "changed.txt"); + var added = Path.Combine(_root, "added.txt"); + var output = Path.Combine(_root, "findings.json"); + File.WriteAllText(list, file + Environment.NewLine); + File.WriteAllText(added, file + Environment.NewLine); + var command = new Command("scan"); + new ScanCommand().Configure(command); + var code = await command.InvokeAsync(["--repository-root", _root, "--changed-files-list", list, + "--added-files-list", added, "--output", output, "--fail-on", "High"]); + Assert.Equal(1, code); + var findings = JsonSerializer.Deserialize>(File.ReadAllText(output))!; + Assert.Equal("DeploymentVersionMismatch", Assert.Single(findings).Rule); + Assert.Equal(file, findings[0].RelativePath); + Assert.Contains("DeploymentVersionMismatch", ReportCommand.BuildContent(output, ReportMode.Detailed, ReportFormat.Html, "Review", "Summary")); + using var sarif = JsonDocument.Parse(ReportCommand.BuildContent(output, ReportMode.Detailed, ReportFormat.Sarif, "Review", "Summary")); + Assert.Equal("DeploymentVersionMismatch", sarif.RootElement.GetProperty("runs")[0].GetProperty("results")[0].GetProperty("ruleId").GetString()); + } + + [Fact] + public void WithoutAnAddedFileList_ScannerDoesNotRevalidateHistoricalVersions() + { + Write("MasterData/2.60.0/data.json"); + Assert.Empty(CmFrameworkScanner.GetFindings(_root, entityCatalog: EntityCatalog.Empty)); + } + + [Fact] + public void OverlappingRoots_ReportProcessRuleAndAddedFileVersionFindingsOnce() + { + var file = Write("DEEs/ProcessRules/EntityTypesAttributes/2.60.0/Rule.cs"); + File.WriteAllText(Path.Combine(_root, file), ProcessRuleValidationTests.Source( + "IAction target = provider.GetService(); target.Delete();", + ns: "EntityTypesAttributes._2_60_0")); + var roots = CodeReviewPaths.ResolveRoots(_root, ReviewScope.DEEs); + + var findings = CmFrameworkScanner.GetFindings(_root, roots: roots.Concat(roots), + changedFiles: [file], addedFiles: [file], entityCatalog: TestEntities.Catalog); + + Assert.Single(findings, finding => finding.Rule == "ProcessRuleUnguardedMutation"); + Assert.Single(findings, finding => finding.Rule == "DeploymentVersionMismatch"); + Assert.All(findings, finding => Assert.Equal(file, finding.RelativePath)); + } + + [Theory] + [InlineData("MasterData/$(version)/**/*.json", "MasterData/2.60.0/data.json")] + [InlineData("MasterData/$(version)/**/*.json", "MasterData/2.60.0/sub/data.json")] + [InlineData("MasterData/2.61.0/*", "MasterData/2.60.0/data.json")] + public void RecursiveAndLiteralVersionSourcesStillDetectWrongFolders(string source, string file) + { + File.WriteAllText(Manifest, System.Text.Json.JsonSerializer.Serialize(new + { + packageType = "Data", version = "2.61.0", + contentToPack = new[] { new { contentType = "MasterData", source } } + })); + Assert.Equal("DeploymentVersionMismatch", Assert.Single(Validate([Write(file)])).Rule); + } + + public void Dispose() => Directory.Delete(_root, true); +} diff --git a/cmf-codereview/tools/DeeCompilerCheck/DeeCompilerCheck.csproj b/cmf-codereview/tools/DeeCompilerCheck/DeeCompilerCheck.csproj new file mode 100644 index 0000000..a7b8a4f --- /dev/null +++ b/cmf-codereview/tools/DeeCompilerCheck/DeeCompilerCheck.csproj @@ -0,0 +1,11 @@ + + + Exe + net8.0 + enable + enable + + + + + diff --git a/cmf-codereview/tools/DeeCompilerCheck/Program.cs b/cmf-codereview/tools/DeeCompilerCheck/Program.cs new file mode 100644 index 0000000..b3c2f50 --- /dev/null +++ b/cmf-codereview/tools/DeeCompilerCheck/Program.cs @@ -0,0 +1,82 @@ +using System.Reflection; +using System.Runtime.InteropServices; +using System.Runtime.Loader; +using System.Text.Json; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.Emit; +// Opt-in compilation check against the platform version restored by a customization project. +// Product assemblies are loaded only in this helper, never by the reviewer's scan. +if (args.Length is < 1 or > 2) +{ + Console.Error.WriteLine("Usage: DeeCompilerCheck [results.json]"); + return 2; +} +var projectPath = Path.GetFullPath(args[0]); +if (!File.Exists(projectPath)) throw new FileNotFoundException("DEE project not found", projectPath); +var assetsPath = Path.Combine(Path.GetDirectoryName(projectPath)!, "obj", "project.assets.json"); +if (!File.Exists(assetsPath)) throw new FileNotFoundException("Restore the DEE project before running this check", assetsPath); +var directory = Path.Combine(Path.GetTempPath(), "dee-compiler-check-" + Guid.NewGuid().ToString("N")); +Directory.CreateDirectory(directory); +var paths = Directory.GetFiles(RuntimeEnvironment.GetRuntimeDirectory(), "*.dll").ToDictionary(p => Path.GetFileName(p)!, p => p, StringComparer.OrdinalIgnoreCase); +using var assets = JsonDocument.Parse(File.ReadAllText(assetsPath)); +var root = assets.RootElement; +var target = root.GetProperty("targets").EnumerateObject().FirstOrDefault(t => !t.Name.Contains('/')); +if (target.Value.ValueKind == JsonValueKind.Undefined) target = root.GetProperty("targets").EnumerateObject().First(); +foreach (var lib in target.Value.EnumerateObject()) +{ + var info = root.GetProperty("libraries").GetProperty(lib.Name); + if (info.GetProperty("type").GetString() != "package") continue; + var group = lib.Value.TryGetProperty("runtime", out var runtime) ? runtime : lib.Value.TryGetProperty("compile", out var compile) ? compile : default; + if (group.ValueKind != JsonValueKind.Object) continue; + foreach (var asset in group.EnumerateObject().Where(a => a.Name.EndsWith(".dll"))) + foreach (var folder in root.GetProperty("packageFolders").EnumerateObject()) + { + var path = Path.Combine(folder.Name, info.GetProperty("path").GetString()!, asset.Name.Replace('/', Path.DirectorySeparatorChar)); + if (File.Exists(path)) paths.TryAdd(Path.GetFileName(path), path); + } +} +AssemblyLoadContext.Default.Resolving += (_, name) => paths.TryGetValue(name.Name + ".dll", out var path) ? AssemblyLoadContext.Default.LoadFromAssemblyPath(path) : null; +var common = AssemblyLoadContext.Default.LoadFromAssemblyPath(paths["Cmf.Foundation.Common.dll"]); +var compiler = common.GetType("Cmf.Foundation.Common.DynamicExecutionEngine.Compiler", true)!; +var utilities = common.GetType("Cmf.Foundation.Common.Utilities", true)!; +var appVersion = utilities.GetProperty("AppVersion")!; +var version = Activator.CreateInstance(appVersion.PropertyType)!; +appVersion.PropertyType.GetProperty("IsOnlyCoreRunning")!.SetValue(version, false); +appVersion.SetValue(null, version); +var allRuntimeReferences = Directory.GetFiles(RuntimeEnvironment.GetRuntimeDirectory(), "*.dll").Where(p => { using var stream = File.OpenRead(p); using var pe = new System.Reflection.PortableExecutable.PEReader(stream); return pe.HasMetadata; }).Select(p => MetadataReference.CreateFromFile(p)).ToArray(); +var customPath = Path.Combine(directory, "Custom.Library.dll"); +var emitted = CSharpCompilation.Create("Custom.Library", [CSharpSyntaxTree.ParseText("namespace Custom.Tools { public class Widget { } }")], allRuntimeReferences, new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)).Emit(customPath); +if (!emitted.Success) throw new Exception(string.Join(";", emitted.Diagnostics)); +paths["Custom.Library.dll"] = customPath; +const BindingFlags flags = BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic; +var generate = compiler.GetMethod("GenerateFullActionCode", flags, null, [typeof(string), typeof(string), typeof(string)], null)!; +var getReferences = compiler.GetMethod("GetReferences", flags)!; +var compileSources = compiler.GetMethods(flags).Single(m => m.Name == "CompileSources" && m.GetParameters()[2].ParameterType == typeof(CSharpCompilationOptions)); +var cases = new[] { + (Name: "missing-assembly", Code: "UseReference(\"\", \"Custom.Tools\"); new Widget();", Expected: false), + (Name: "missing-namespace", Code: "UseReference(\"Custom.Library.dll\", \"\"); new Widget();", Expected: false), + (Name: "valid", Code: "UseReference(\"Custom.Library.dll\", \"Custom.Tools\"); new Widget();", Expected: true), + (Name: "independent-inputs", Code: "UseReference(\"System.Console.dll\", \"Custom.Tools\"); UseReference(\"Custom.Library.dll\", \"\"); new Widget();", Expected: true), + (Name: "empty-no-op", Code: "UseReference(\"\", \"\");", Expected: true) +}; +var checks = new List(); +foreach (var item in cases) +{ + var source = (string)generate.Invoke(null, [item.Name, item.Code, "return true;"])!; + var referenceNames = (IEnumerable)getReferences.Invoke(null, [item.Code + "return true;"])!; + var references = referenceNames.Select(p => paths.TryGetValue(Path.GetFileName(p), out var resolved) ? resolved : p).Distinct().ToArray(); + var result = (EmitResult)compileSources.Invoke(null, [new[] { source }, references, new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary), directory, item.Name + ".dll"])!; + var errors = result.Diagnostics.Where(d => d.Severity == DiagnosticSeverity.Error).Select(d => d.ToString()).ToArray(); + checks.Add(new { item.Name, item.Expected, result.Success, Errors = errors }); + if (result.Success != item.Expected) throw new Exception(JsonSerializer.Serialize(checks)); +} +var json = JsonSerializer.Serialize(new { ProductAssembly = common.Location, Version = common.GetName().Version!.ToString(), ArtifactDirectory = directory, Checks = checks }, new JsonSerializerOptions { WriteIndented = true }); +if (args.Length == 2) +{ + var output = Path.GetFullPath(args[1]); + Directory.CreateDirectory(Path.GetDirectoryName(output)!); + File.WriteAllText(output, json); +} +Console.WriteLine(json); +return 0;