### **Narrative Dynamics Language (NDL) Guide** #### **1. Philosophy & Purpose** The Narrative Dynamics Language (NDL) is a high-level, declarative query language enabling the analysis of information evolution. It is primary interface for querying dynamics within Research Tool. Traditional query languages can find states (e.g., "What was the company's risk level in Q2?"). NDL is built to analyze the **dynamics between states** (e.g., "How fast is the risk level accelerating, is that analysis verified, and who disagrees with it?"). NDL bridges the gap between high-level semantic analysis and low-level evidentiary proof. It empowers analysts not only to understand the narrative but also to **trace every insight back to its precise origin** within the source material. Every query is an exploration of trustworthy, auditable, and context-rich insights. To that end, NDL provides commands not only for analyzing dynamics (`SHOW`) and tracing evidence (`TRACE`), but also for directly querying the rich semantic and collaborative layers of the graph. The `FIND` command empowers analysts to explore the network of human insight itself—discovering questions, validating corrections, and auditing the verification process that underpins the system's trustworthiness. Finally, by directly leveraging the ontology's distinction between evidence and templates, NDL can perform conformance analysis, comparing documents against a standardized baseline to instantly flag deviations. Beyond analyzing the dynamics of subjects, NDL is uniquely capable of analyzing the **dynamics of insight itself**. By leveraging the rich, machine-readable properties within `:Annotation` nodes, analysts can quantify and track the evolution of expert opinion, sentiment, and consensus over time. This transforms the social and collaborative layer from a simple commentary stream into a queryable, quantitative dimension. Queries can now answer not just "How is risk accelerating?" but also "Is our team's confidence in this assessment growing or waning, and at what rate?" This fulfills the vision of the Research Tool: to create a living knowledge map where the act of analysis is as measurable and auditable as the data being analyzed. #### **2. Core Analytical Concepts** To understand NDL, you must first understand the building blocks of the analytical graph it operates on. NDL commands are designed to manipulate these specific concepts. 1. **The Source Digital Twin:** The foundational layer of evidence. It is a high-fidelity replica of all source documents, composed of: * **:SourceVersion:** A specific, concrete instance of a document at a point in time (e.g., "Q2 2025 10-K"). * **:Element:** An atomic piece of the document (a paragraph, a table cell, a section header). The hierarchy and sequence of elements are preserved. 2. **The `:Change` Node:** The objective record of a difference. When two `:SourceVersion`s are compared, the system automatically generates `:Change` nodes to flag the raw, mechanical differences (`ADDITION`, `DELETION`, `MODIFICATION`). This is the "what" of the change. 3. **:State Node:** A snapshot of a subject's condition at a point in time. This is typically an `:Event` node (e.g., a specific "MRI Scan") or a `:SourceVersion` (e.g., a "Q2 10-K Filing"). It represents the "what" at a specific "when." 4. **:Analytic Spine:** A pre-materialized, queryable axis that provides context. NDL can query along three types of spines: * **Temporal Spine (`:Day`, `:Month`, `:Year`):** Provides the "when." * **Geospatial Spine (`:GeoCell`):** Provides the "where." * **Thematic Spine (e.g., `:SeverityLevel`):** Provides the qualitative or quantitative context (the "how bad," "how much," or "what kind"). 5. The **Annotation Spine:** A distinct type of analytical axis used for quantifying and tracking human-generated insights over time. Unlike pre-materialized `Temporal` or `Thematic` spines, an `Annotation Spine` is constructed on-the-fly from the data within `:Annotation` nodes themselves. * **How it Works:** When a query uses an `Annotation Spine`, NDL gathers all `:Annotation` nodes related to the query's subject. It then uses the `classification_spine` property to select the relevant annotations (e.g., all annotations where `classification_spine = 'AnalystConfidence'`) and uses their `classification_value` and `createdAt` timestamp to construct a time series. * **Purpose:** This allows NDL to perform dynamic analysis (e.g., calculating trends, consensus, volatility) on the evolution of expert judgments, treating analyst opinion as a measurable data stream. 5. **:AnalyticArc Node:** This is the heart of NDL. It is the explicit, pre-computed analysis of the change **between two :State nodes.** Crucially, it is a **node**, not merely a relationship. This allows it to hold rich metadata, such as: * `type`: The semantic meaning of the change (e.g., `growth`, `escalation`, `acceleration`). * `value`: The magnitude of the change. * `change_in_rate`: The acceleration or deceleration. * `verified`: A boolean indicating if a human expert has validated this analysis. * `competes_with`: A list of other `AnalyticArc` IDs that offer a competing analysis of the same event. 7. **:Annotation & Enrichment:** These nodes provide the human-generated and AI-proposed semantic layer. An `:Annotation` captures expert rationale, while an `:Enrichment` (like an `:Entity` or `:Topic`) tags the data. NDL uses this layer to provide deep, contextual filtering. Using the `FIND` command, these nodes can also be queried directly as the primary subjects of an investigation. 8. **The Entity Network:** Beyond individual enrichments, the Research Tool supports a web of connections between entities (e.g., People, Companies, Products). Relationships like `(:Person)-[:IS_CEO_OF]->(:Company)` or `(:Company)-[:IS_SUPPLIER_OF]->(:Product)` form a rich, queryable network. NDL provides tools to traverse this network, uncover hidden relationships, and understand the structural context connecting different actors. #### **3. NDL Command Reference** **A. The `DESCRIBE` Command** * **Purpose:** Used to explore the available analytical contexts. It makes the pre-materialized spines discoverable. * **Thematic Spine Syntax:** ```ndl DESCRIBE THEMATIC SPINE ``` * **Example:** Before analyzing patient symptoms, an analyst can discover what severity levels are even possible to query. ```ndl -- Discovers all defined levels in the "Symptom Severity" spine. DESCRIBE THEMATIC SPINE "Symptom Severity" ``` * **Returns:** A list of the spine's nodes and their ranks (e.g., {name: "Mild", rank: 1}, {name: "Moderate", rank: 2}, {name: "Severe", rank: 3}). * **Geospacial Spine Syntax:** ```ndl DESCRIBE GEOSPATIAL SPINE [ (LEVELS | NAMED_REGIONS [AT LEVEL ]) ] ``` * **Description:** Allows an analyst to explore the structure and contents of the pre-materialized `:GeoCell` hierarchy. * `LEVELS`: Returns the range of S2 levels that have been pre-materialized in the graph (e.g., from level 4 to level 14). This helps an analyst understand the available resolutions for analysis. * `NAMED_REGIONS`: Returns a list of all `:GeoCell` nodes that have a human-readable `name` property (e.g., "California", "Paris", "Midtown"). This provides a user-friendly entry point for queries. * `AT LEVEL `: An optional filter for `NAMED_REGIONS` to find named areas at a specific S2 resolution. * **Examples:** * **Find the available analytical resolutions:** ```ndl DESCRIBE GEOSPATIAL SPINE LEVELS ``` * **Returns:** A result like `{min_level: 4, max_level: 14}`. * **Discover all named locations in the system:** ```ndl DESCRIBE GEOSPATIAL SPINE NAMED_REGIONS ``` * **Returns:** A list of named cells, e.g., `[{name: "USA", level: 4, s2_token: "809"}, {name: "California", level: 6, s2_token: "89c2"}, {name: "Paris", level: 8, s2_token: "48761"}]`. * **Find only the named countries (assuming they are stored at a specific level):** ```ndl DESCRIBE GEOSPATIAL SPINE NAMED_REGIONS AT LEVEL 4 ``` **B. The `SHOW` Command** * **Purpose:** The main analytical engine of NDL. It performs dynamic calculations on the history of one or more subjects. * **Syntax:** ```ndl SHOW FOR [ON (THEMATIC | TEMPORAL | GEOSPATIAL | ANNOTATION) SPINE ] [CONTEXT ] [STATUS ] [WHERE ] [SHOWING (, ...)] ``` * **Clauses:** * `SHOW `: (Required) The dynamic calculation to perform (e.g., `VELOCITY`, `ACCELERATION`, `DISAGREEMENT`). See the Metrics Reference below. * `FOR `: (Required) Specifies the primary subject(s) of the analysis (e.g., `(c:Company {ticker: "AAPL"})`). * `ON SPINE `: (Optional) Specifies the analytical axis for the calculation. `` can be `THEMATIC`, `TEMPORAL`, or `GEOSPATIAL`. For `GEOSPATIAL`, the syntax is expanded to provide hierarchical and proximity-based filtering. Required by most quantitative metrics. * **Example For `THEMEATIC` Spine:** ```ndl `ON THEMATIC SPINE "CorporateRisk"` ``` * **Example for `ANNOTATION` Spine:** ```ndl `ON ANNOTATION SPINE "AnalystConfidence"` ``` * **Description for `ANNOTATION`:** When `ANNOTATION` is used as the ``, the `SHOW` command pivots its analysis. Instead of tracking the state of the `AnchorPattern` itself, it tracks the `classification_value` of `:Annotation` nodes that relate to the anchor. The `` must correspond to a `classification_spine` value stored on the `:Annotation` nodes. This powerful clause enables direct analysis of trends in expert opinion, confidence, or other human-generated quantitative judgments. * **Example For `GEOSPATIAL` Spine:** ```ndl ON GEOSPATIAL SPINE [AT LEVEL ] [WITHIN ] [NEAR {lat: , lon: } [RADIUS ]] ``` GEOSPACIAL SPIINE clauses can be combined for multi-faceted geospatial filtering: * `AT LEVEL `: **(Aggregation & Resolution Control)** Constrains the analysis to `:GeoCell` nodes at a specific S2 level or a list of levels. This is the primary mechanism for "rolling up" data to a high-level view (e.g., `AT LEVEL 6` for states) or "drilling down" to a granular one (e.g., `AT LEVEL 14` for city blocks). * `WITHIN `: **(Hierarchical Filtering)** Narrows the analysis to locations inside a specific parent geographic area. The `` can be a user-friendly named region or a precise S2 token. This directly leverages the `PARENT_OF` relationships in the spine. * Example (by name): `WITHIN (gc:GeoCell {name: "California"})` * Example (by token): `WITHIN (gc:GeoCell {s2_token: "89c259"})` * `NEAR {lat, lon} [RADIUS ]`: **(Proximity Search)** A user-friendly search that finds events around a specific point. * **How It Works:** This clause is a high-level convenience. The NDL engine translates this query into an efficient, spine-based operation: 1. It uses the S2 library to calculate the set of `:GeoCell` tokens that "cover" the specified radius at an appropriate `level`. 2. It then executes a fast, index-backed graph query to find all data linked to those specific `:GeoCell` nodes. * This approach converts an expensive, on-the-fly radius calculation into a simple, performant graph traversal, perfectly aligning with Research Tool's philosophy of pre-computation. * `CONTEXT `: (Optional) A filter to narrow the analysis to states or changes related to a specific entity, topic, or piece of evidence. This is used to query the semantic enrichment layer. * **Example:** `CONTEXT (topic:Topic {topic_type: "disagreement"})-[:RELATES_TO]->(state)` * `STATUS `: (Optional) Filters the `AnalyticArcs` based on their human-in-the-loop validation status. `` can be `VERIFIED`, `UNVERIFIED`, or `CONTESTED` (arcs that have competing analyses). * **Example:** `STATUS VERIFIED` * `WHERE `: (Optional) Applies standard filters to the final results, such as time windows or value thresholds. * **Example:** `WHERE result.value > 50 AND result.date IN LAST 90 DAYS` * `AGAINST `: (Required for `DEVIATION` metric) Specifies the `:Boilerplate` source that the `AnchorPattern` should be compared against. This is the key component for conformance and compliance analysis. * **Example:** `AGAINST (b:Boilerplate {name: "PMO Report Template"})` * `SHOWING (, ...)`: (Optional, for `DEVIATION` metric) A filter to show only specific types of changes, mirroring the functionality of the `COMPARE` command. `` can be `ADDITION`, `DELETION`, or `MODIFICATION`. * **Example:** `SHOWING (MODIFICATION, ADDITION)` **C. The `FIND` Command** * **Purpose:** Used to directly query and retrieve sets of nodes from the graph based on their properties and relationships. Unlike `SHOW`, which returns a calculated metric, `FIND` returns the nodes themselves. This is the primary command for exploring the Social Layer (Pillar 3) and the semantic enrichment layer. * **Syntax:** ```ndl FIND [AS ] [FOR ] [RELATING VIA ] [WHERE ] [ORDER BY [ASC|DESC]] [LIMIT ] ``` * **Clauses:** * `FIND [AS ]`: (Required) The type of node to retrieve, corresponding to a label in the ontology (e.g., `ANNOTATIONS`, `ENTITIES`, `EVENTS`, `ANALYTIC_ARCS`). The optional alias is used in other clauses. * `FOR `: (Optional) Establishes the context for the search, specifying the node or nodes the target is related to. * **Example:** `FOR (c:Company {ticker: "AAPL"})` * `RELATING VIA `: (Optional, but required if `FOR` is used) Explicitly defines the connection between the `` and the ``. The pattern must use the alias for the target node and the alias from the anchor pattern. * **Example:** `RELATING VIA (ann)-[:ANNOTATES]->(c)` * `WHERE `: (Optional) A standard filter clause to refine results based on the properties of the retrieved nodes or related nodes. * **Example:** `WHERE ann.annotation_type = 'question' AND ann.status = 'proposed'` * `ORDER BY` and `LIMIT`: (Optional) Standard clauses for sorting and paginating results. * **Key Use Cases for `FIND`:** 1. **Auditing & Collaboration:** Finding all annotations of a certain type, by a specific user, or with a given status. 2. **Targeted Discovery:** Retrieving specific entities or events that are linked to a piece of commentary. 3. **Verification Workflows:** Finding insights (like `AnalyticArcs`) that are justified by annotations with a 'proposed' status, enabling a review and approval process. * **Examples:** * **Find all open questions asked by a specific analyst:** ```ndl -- Directly queries the Annotation nodes themselves FIND ANNOTATIONS AS ann WHERE ann.createdBy = 'analyst_jane' AND ann.annotation_type = 'question' AND ann.status = 'proposed' ORDER BY ann.createdAt DESC ``` * **Find all `AnalyticArcs` justified by a proposed 'correction' annotation:** This query bridges Pillar 2 and Pillar 3, finding analytical insights that are pending verification. ```ndl -- The target is the AnalyticArc, filtered by its related Annotation FIND ANALYTIC_ARCS AS arc RELATING VIA (arc)-[:JUSTIFIED_BY]->(ann) WHERE ann.annotation_type = 'correction' AND ann.status = 'proposed' ``` * **Find all annotations attached to a specific section of a document:** ```ndl -- The anchor is the Element, the target is the Annotation FIND ANNOTATIONS AS ann FOR (el:Element {unique_id: "section_4_para_2_v2"}) RELATING VIA (ann)-[:ANNOTATES]->(el) ``` * **Find all 'key_insight' annotations related to 'supply chain' topics:** ```ndl FIND ANNOTATIONS AS ann FOR (t:Topic {name: "supply chain"}) RELATING VIA (ann)-[:ANNOTATES]->(t) WHERE ann.annotation_type = 'key_insight' ``` **D. The `FIND PATH` Command** * **Purpose:** Directly addresses the need for network analysis within NDL. This command is designed to discover and return the sequence of nodes and relationships that connect two or more entities in the graph. It is the primary tool for exploring the semantic entity network, answering questions like "How is Person X connected to Company Y?" or "What chains of influence exist around a specific topic?" * **Syntax:** ```ndl FIND [ (SHORTEST | ALL) ] PATH AS FROM TO [VIA ] [MAX_DEPTH ] [WHERE ] [ORDER BY [ASC|DESC]] [LIMIT ] ``` * **Clauses:** * `FIND [ (SHORTEST | ALL) ] PATH AS `: (Required) Declares the intent to find a path. * `SHORTEST`: (Default) Instructs the engine to find the path(s) with the fewest number of relationships. This is best for finding the most direct connection. * `ALL`: Instructs the engine to find all possible paths that meet the criteria, up to the `MAX_DEPTH`. This is used for comprehensive network exploration. * `AS `: (Required) Assigns a name to the path object, which can be used in other clauses like `WHERE` or `ORDER BY`. * `FROM `: (Required) Specifies the starting node for the path search (e.g., `(p:Person {name: "John Doe"})`). * `TO `: (Required) Specifies the ending node for the path search (e.g., `(c:Company {name: "ACME Corp"})`). * `VIA `: (Optional) Constrains the search to only traverse specific types of relationships. This is crucial for focused analysis. The specifier can be a single type, a list, or use wildcards. * **Example (Single Type):** `VIA 'IS_CEO_OF'` * **Example (List):** `VIA ('IS_SHAREHOLDER_OF', 'IS_BOARD_MEMBER_OF')` * **Example (Wildcard):** `VIA 'IS_*_OF'` - This powerfully leverages the ontology to traverse any defined entity relationship. * **Example (Directionless):** `VIA *` - Traverses any relationship type in any direction. * `MAX_DEPTH `: (Optional, but recommended for `ALL PATHS`) A performance and safety control that limits the maximum length (number of relationships) of the paths to find. This prevents runaway queries in highly connected graphs. * `WHERE `: (Optional) A standard filter clause to refine results based on properties of the *path itself* or the nodes and relationships *within the path*. * **Example (Filter by Path Length):** `WHERE length(p) > 2` * **Example (Filter by Node Property in Path):** `WHERE all(n in nodes(p) WHERE n.verified = true)` * `ORDER BY` and `LIMIT`: (Optional) Standard clauses for sorting and paginating the returned paths, often used with `length()`. * **Key Use Cases & Examples:** * **Discover a direct connection:** ```ndl -- Find the shortest connection path between Person X and Company Y. FIND SHORTEST PATH AS p FROM (e1:Entity {name:"Person X"}) TO (e2:Entity {name:"Company Y"}) VIA 'IS_*_OF' -- Use the semantic entity relationships ``` * **Compliance: Check for conflicts of interest:** ```ndl -- Find all financial connections between a regulator and a supervised company. FIND ALL PATHS AS financial_links FROM (reg:Person {name: "Regulator Jane"}) TO (comp:Company {ticker: "FINCORP"}) VIA ('IS_SHAREHOLDER_OF', 'IS_BOARD_MEMBER_OF', 'HAS_SPOUSE_WHO_IS_EMPLOYEE_OF') MAX_DEPTH 3 ``` * **Investigative Analysis: Uncover hidden networks:** ```ndl -- Start with a person of interest and discover all other people they are connected to -- within 3 steps, and show how they are connected. FIND ALL PATHS AS network FROM (p1:Person {id: "person_of_interest_123"}) TO (p2:Person) -- To any other person VIA * MAX_DEPTH 3 WHERE p1 <> p2 -- Ensure the start and end nodes are different ``` * **Advanced Filtering: Find paths through verified entities only:** ```ndl -- Find the path from a supplier to a product, but only if every -- entity along that path has been verified by an analyst. FIND PATH AS p FROM (s:Company {name: "Global Parts Inc."}) TO (prod:Product {name: "Widget Pro"}) WHERE all(node in nodes(p) WHERE node.verified = true) ``` #### **4. NDL Metrics Reference (for `SHOW` command)** * `VELOCITY` * **Description:** Calculates the average rate of change (magnitude / time) along a specified spine. * **Requires:** `ON THEMATIC SPINE` clause. * **Example:** `SHOW VELOCITY FOR (p:Patient {id: "P-0877"}) ON THEMATIC SPINE "TumorVolumeCM3"` * **Returns:** The average change per unit of time (e.g., "1.2 cm³/month"). * `ACCELERATION` * **Description:** Calculates the rate of change of velocity, answering "Is the situation changing faster or slower over time?" * **Requires:** `ON THEMATIC SPINE` clause. * **Example:** `SHOW ACCELERATION FOR (c:Company {id: "C-123"}) ON THEMATIC SPINE "DefectRate" STATUS VERIFIED` * **Returns:** The average acceleration (e.g., "+0.5 defects/month²"). * `STATE_DURATION` * **Description:** Calculates the total time a subject spent in a specific qualitative state. * **Requires:** The metric must specify the state name. * **Example:** `SHOW STATE_DURATION 'Stable' FOR (r:Region {name: 'Arida'})` * **Returns:** The total duration in days. * `VOLATILITY` * **Description:** Measures the degree of fluctuation or instability along a spine, regardless of direction. * **Requires:** `ON THEMATIC SPINE` clause. * **Example:** `SHOW VOLATILITY FOR (s:Stock {ticker:"NVDA"}) ON THEMATIC SPINE "PriceUSD"` * **Returns:** A numerical score representing the standard deviation or variance of changes. * `DISAGREEMENT` * **Description:** A unique NDL metric that directly queries the social/collaborative layer. It finds all instances where multiple, competing `AnalyticArcs` exist for the same change event. * **Requires:** No `ON SPINE` clause needed. * **Example:** `SHOW DISAGREEMENT FOR (report:CanonicalSource {name: "Country X Analysis"})` * **Returns:** A list of contested events, the competing `AnalyticArc` interpretations for each, and the annotations justifying them. * `DEVIATION` * **Description:** A powerful compliance and conformance metric that identifies all differences between a target document and a specified `:Boilerplate` baseline. It operates on `:Change` nodes that are linked to `:Boilerplate` elements via the `DEVIATES_FROM` relationship. This is used to answer questions like, "How does this contract differ from our standard template?" or "Is this report missing any required sections?" * **Requires:** `AGAINST ` clause. * **Example:** `SHOW DEVIATION FOR (cs:CanonicalSource {source_id: "Contract_789_v2"}) AGAINST (b:Boilerplate {name: "Standard_NDA_Template_2025"})` * **Returns:** A list of the `:Change` nodes representing the deviations, including the `change_type` and the content of the modified element. #### **Metrics for Annotation Spines** These metrics are designed specifically for use with `ON ANNOTATION SPINE`. * `TREND` * **Description:** Calculates the rate of change of annotation values over time, effectively measuring the trajectory of opinion or assessment. It answers, "Is sentiment trending positive or negative, and how fast?" * **Requires:** `ON ANNOTATION SPINE` clause. * **Example:** `SHOW TREND FOR (p:Project {id:"Chimera"}) ON ANNOTATION SPINE "RiskAssessment"` * **Returns:** A numerical value representing the slope of the trend line (e.g., "+0.5 points/month"). * `CONSENSUS` * **Description:** Measures the level of agreement among analysts. It calculates the standard deviation of `classification_value` for all relevant annotations within a given time window. * **Requires:** `ON ANNOTATION SPINE` clause. * **Example:** `SHOW CONSENSUS FOR (t:Topic {name:"Market Viability"}) ON ANNOTATION SPINE "ViabilityScore"` * **Returns:** A numerical score. A lower score indicates higher consensus (less deviation in scores), while a higher score indicates significant disagreement. * `INTENSITY` * **Description:** Measures the volume and frequency of annotations over time. This is useful for identifying "chatter" or periods of increased analytical activity around a subject. * **Requires:** `ON ANNOTATION SPINE` clause. * **Example:** `SHOW INTENSITY FOR (c:Company {ticker:"ACME"}) ON ANNOTATION SPINE "GeneralCommentary"` * **Returns:** A count of annotations per unit of time (e.g., "15 annotations/week"). ### **The `TRACE` Command** * **Purpose:** Navigates the structural hierarchy of a source document or traces an analytical finding back to its evidentiary source. * **Syntax:** ```ndl TRACE FOR [DEPTH ] ``` * **Trace Modes:** * `HIERARCHY`: Displays the structural layout of a document (`:SourceVersion`) by following `PARENT_OF` relationships between its `:Element`s. * `PROVENANCE`: Starts from a high-level insight (like an `:Event` or `:AnalyticArc`) and traces back to the specific `:Element`(s) that act as its evidence via the `EVIDENCED_BY` relationship. * **Examples:** * **Trace the structure of a financial filing:** ```ndl -- Returns the section and paragraph hierarchy for the Q2 2025 10-K TRACE HIERARCHY FOR (sv:SourceVersion {name: "AAPL 10-K for Q2 2025"}) DEPTH 3 ``` * **Trace an event back to its source text:** ```ndl -- Finds the specific paragraph in an MRI report that mentions "gait instability" TRACE PROVENANCE FOR (event:Event {id: "gait_instability_event_456"}) ``` ### **The `COMPARE` Command** * **Purpose:** Directly queries the raw, machine-detected `:Change` nodes to identify objective differences between document versions. This is used for auditing, verification, and discovering changes that have not yet been analyzed. * **Syntax:** ```ndl COMPARE [WITH ] FOR [SHOWING (, ...)] ``` * **Clauses:** * ``: Can be a version number (e.g., `VERSION 1`), `LATEST`, or a specific `unique_id`. If `WITH` is omitted, it compares the specified version to its immediate predecessor. * `FOR`: (Required) Specifies the conceptual document being compared (e.g., `(cs:CanonicalSource {name: "Patient P-0877 MRI Series"})`). * `SHOWING`: (Optional) A filter to show only specific types of changes. `ChangeType` can be `ADDITION`, `DELETION`, `MODIFICATION`, or `MOVE`. * **Examples:** * **Find all new and deleted text between two MRI report versions:** ```ndl COMPARE VERSION 1 WITH VERSION 2 FOR (cs:CanonicalSource {source_id: "P-0877_MRI_SERIES"}) SHOWING (ADDITION, DELETION) ``` * **Show what was modified in the latest 10-K filing compared to the previous one:** ```ndl COMPARE LATEST FOR (cs:CanonicalSource {name: "AAPL 10-K Filings"}) SHOWING (MODIFICATION) ``` #### **4. Comprehensive Examples** The following examples now show how an analyst can combine the different NDL commands for a complete workflow. **Example 1: Verified Acceleration of Corporate Risk** * **Question:** "Show me the verified rate at which 'supply chain' risk has been accelerating for company 'C-123' over the last year." * **NDL Query:** ```ndl SHOW ACCELERATION FOR (c:Company {id: "C-123"}) ON THEMATIC SPINE "CorporateRisk" CONTEXT (topic:Topic {name: "supply chain"})-[:RELATES_TO]->(state) STATUS VERIFIED WHERE state.effective_date > "2024-07-30" ``` **Example 2: Geospatial Proximity Search for High Severity Events** * **Question:** "Find all patient-reported 'gait instability' events of 'Severe' severity that occurred in the last quarter within 50km of a specific clinic." * **NDL Query:** ```ndl SHOW LATEST_STATE FOR (p:Patient) ON GEOSPATIAL SPINE NEAR {lat: 37.7749, lon: -122.4194} RADIUS 50 -- Clinic Lon/Lat CONTEXT (event:Event)-[:IS_INSTANCE_OF]->(:Symptom {name: "gait instability"}), (event)-[:HAS_SEVERITY]->(:SeverityLevel {name: "Severe"}) WHERE event.effective_date IN LAST 90 DAYS ``` * **Explanation:** This query is now clearer. The `NEAR...RADIUS` clause provides an intuitive way to ask a proximity-based question, while the system guarantees high performance by translating it into an efficient S2 cell covering query against the pre-materialized `:GeoCell` nodes. ``` **Example 3: Finding Expert Disagreement** * **Question:** "In our analysis of intelligence reports on 'Region Arida', where do our experts disagree on the interpretation of stability changes?" * **NDL Query:** ```ndl SHOW DISAGREEMENT FOR (r:Region {name: "Arida"}) CONTEXT (state)-[:EVIDENCED_BY]->(:Element)<-[:CONTAINS]-(:SourceVersion)<-[:HAS_VERSION]-(:CanonicalSource {source_id: "intel_reports_arida"}) ``` * **Explanation:** This query looks for all `AnalyticArcs` describing the state changes of the 'Arida' region. The `CONTEXT` clause limits the search to only those states evidenced by the specified intelligence reports. The `SHOW DISAGREEMENT` metric then filters these results to only return the moments in time where `competing_with` relationships exist, instantly highlighting points of analytical conflict for review. **Example 4: A Complete Collaborative Analysis Workflow** This workflow demonstrates how an entire team uses NDL to move from raw data to verified, collaborative insight. 1. **Step 1: Discover a Change (Auditor)** An auditor checks for modifications in the latest risk report for "Region Arida". ```ndl COMPARE LATEST FOR (cs:CanonicalSource {source_id: "intel_reports_arida"}) SHOWING (MODIFICATION) ``` 2. **Step 2: Ask a Question about the Change (Analyst)** The `COMPARE` command flags a modified paragraph (Element `e-987-v2`). An analyst reviews it and is unsure about the source of the new data. They add a question annotation in the system. 3. **Step 3: Find and Review Open Questions (Team Lead)** The team lead runs a daily query to find all unresolved questions from their team. ```ndl -- The FIND command is perfect for this workflow task FIND ANNOTATIONS AS ann WHERE ann.annotation_type = 'question' AND ann.status = 'proposed' ORDER BY ann.createdAt ASC ``` 4. **Step 4: Find the Justification for the Contested Analysis (Team Lead)** The lead sees the question about Element `e-987-v2`. The system has already generated a provisional `AnalyticArc` for this change. The lead wants to see the justification for that arc before responding. ```ndl -- Find the specific annotation justifying the arc related to the element in question FIND ANNOTATIONS AS ann FOR (arc:AnalyticArc) RELATING VIA (arc)-[:JUSTIFIED_BY]->(ann), (arc)-[:HAS_STATE_AFTER]->(:State)-[:EVIDENCED_BY]->(el:Element {unique_id: "e-987-v2"}) ``` 5. **Step 5: Validate the Insight and Finalize Analysis (Team Lead & System)** After reviewing, the lead confirms the change is valid and creates a `validation` annotation. This action in the UI automatically sets the `verified` flag on the `AnalyticArc` to `true` and updates the status of the analyst's original question to `resolved`. Now, the insight is part of the trusted knowledge base. 6. **Step 6: Run the Final Dynamic Analysis (Director)** A director can now confidently query the verified acceleration of risk, knowing the underlying data has been through a rigorous, auditable, human-in-the-loop validation process. ```ndl SHOW ACCELERATION FOR (r:Region {name: "Arida"}) ON THEMATIC SPINE "StabilityRisk" STATUS VERIFIED -- This now correctly filters for the validated insight ``` **Example 5: Auditing for Compliance Deviations** * **Question:** "I need to audit the latest 'Q3 Financial Statement' for ACME Corp. Show me all sections that were modified or added compared to our official 'SEC 10-Q Filing Template', as I need to check them for non-standard language." * **NDL Query:** ```ndl SHOW DEVIATION FOR (sv:SourceVersion {name: "ACME Corp Q3 Financial Statement"}) AGAINST (b:Boilerplate {name: "SEC 10-Q Filing Template"}) SHOWING (MODIFICATION, ADDITION) ``` * **Explanation:** 1. `SHOW DEVIATION`: Specifies the core analytical goal—to find differences from a baseline. 2. `FOR (...)`: Identifies the specific document version we want to audit. 3. `AGAINST (...)`: This crucial clause tells NDL which template to use as the ground truth. The query engine will now look for `:Change` nodes that connect elements from the "ACME Corp Q3 Financial Statement" to elements within the "SEC 10-Q Filing Template" via the `DEVIATES_FROM` relationship. 4. `SHOWING (...)`: This optional filter narrows the results to only the most relevant change types for this task—new text (`ADDITION`) and altered text (`MODIFICATION`), while ignoring deletions. * **Result:** The query returns a precise, actionable list of every element in the Q3 statement that contains non-standard language, allowing an auditor to focus their attention immediately without having to perform a manual, line-by-line comparison. **Example 6: A Full Geospatial Analysis Workflow (Roll-up and Drill-down)** * **Scenario:** An epidemiologist is analyzing symptom report data for the United States. They want to start with a high-level view and then investigate hotspots. * **Step 1: Discover Available Geographic Context (Analyst)** First, the analyst checks what named regions exist to get their bearings. ```ndl DESCRIBE GEOSPATIAL SPINE NAMED_REGIONS AT LEVEL 6 ``` * **Result:** A list including `{name: "California", level: 6, s2_token: "89c2"}`. * **Step 2: Perform a High-Level "Roll-up" Analysis (Analyst)** The analyst wants to see the total number of "fever" reports aggregated at the state level (S2 level 6) across the entire country. ```ndl SHOW COUNT FOR (event:Event)-[:IS_INSTANCE_OF]->(:Symptom {name: "fever"}) ON GEOSPATIAL SPINE AT LEVEL 6 CONTEXT (event)-[:OCCURRED_AT]->(gc:GeoCell), (gc)-[:PARENT_OF*]->(usa:GeoCell {name: "USA"}) -- Ensure we only look inside the US ``` * **Explanation:** The `AT LEVEL 6` clause aggregates all symptom reports into their corresponding level 6 `:GeoCell` nodes, providing a state-by-state count. * **Step 3: "Drill-down" into a High-Impact Region (Analyst)** The previous query reveals a high number of reports in California. The analyst now drills down to a neighborhood view (level 12) *only within California* to find specific hotspots. ```ndl SHOW COUNT FOR (event:Event)-[:IS_INSTANCE_OF]->(:Symptom {name: "fever"}) ON GEOSPATIAL SPINE AT LEVEL 12 WITHIN (gc:GeoCell {name: "California"}) WHERE event.effective_date IN LAST 30 DAYS ``` * **Explanation:** This query perfectly demonstrates the power of combining clauses. `WITHIN` first narrows the search space to only cells that are children of the "California" `:GeoCell`, and `AT LEVEL 12` sets the final reporting granularity to the neighborhood level. This is a highly efficient query that would be impossible without the pre-materialized spine. **Example 7: Combining Dynamic Analysis with Network Discovery** * **Scenario:** An analyst at a financial regulator is alerted to potential market manipulation. Their goal is to identify the key actors and determine if they have any undeclared connections. * **Step 1: Identify Anomalous Activity (Dynamic Analysis)** The analyst first looks for companies exhibiting extreme, verified volatility in their stock price. ```ndl SHOW VOLATILITY FOR (c:Company) ON THEMATIC SPINE "StockPrice" STATUS VERIFIED WHERE result.date IN LAST 30 DAYS ORDER BY result.value DESC LIMIT 5 ``` * **Result:** The query returns two companies, "QuantumLeap Inc." and "StellarDrive LLC," with unusually high volatility. * **Step 2: Find the People Discussing These Companies (Enrichment Querying)** Next, the analyst wants to find who has been recently writing or asking questions about these two companies. ```ndl FIND ENTITIES AS p FOR (c:Company) RELATING VIA (ann:Annotation)-[:ANNOTATES]->(c) <-[:APPEARS_IN]-(p:Person) WHERE (c.name = 'QuantumLeap Inc.' OR c.name = 'StellarDrive LLC') AND ann.annotation_type IN ('question', 'key_insight') ``` * **Result:** The query identifies three people: "Analyst Bob," "Trader Alice," and "Executive Carol." * **Step 3: Discover Hidden Connections (Network Traversal)** This is the crucial step. The analyst now uses `FIND PATH` to see if any of these people, who are all focused on the same anomalous companies, have a pre-existing professional or financial relationship that was not immediately obvious. ```ndl -- Is the Executive connected to the Trader? FIND SHORTEST PATH AS p FROM (p1:Person {name: "Executive Carol"}) TO (p2:Person {name: "Trader Alice"}) VIA ('IS_SHAREHOLDER_OF', 'IS_BOARD_MEMBER_OF', 'IS_FORMER_COLLEAGUE_OF') MAX_DEPTH 4 ``` * **Explanation:** This query searches for a professional or financial link between the two individuals. It doesn't look for just a direct link but will search up to 4 degrees of separation. * **Result:** The query returns a path: `(Executive Carol) -> [:IS_BOARD_MEMBER_OF] -> (Charity Foundation Z) <- [:IS_BOARD_MEMBER_OF] <- (Trader Alice)`. They sit on the same charity board, a non-obvious connection that warrants further investigation. * **Conclusion:** This workflow demonstrates the power of NDL's layered approach. The analyst seamlessly moved from a **dynamic** query (`SHOW VOLATILITY`), to a **semantic** query (`FIND ENTITIES`), and finally to a **network** query (`FIND PATH`) to uncover a critical, actionable insight that would be missed by other query languages. **Example 8: Tracking the Trend of Analyst Risk Assessment** **Question:** "How has the team's assessment of risk for 'Project Chimera' evolved over the last quarter? Is it trending up or down?" **NDL Query:** ```ndl SHOW TREND FOR (p:Project {name: "Project Chimera"}) ON ANNOTATION SPINE "ProjectRiskScore" -- Corresponds to annotation.classification_spine WHERE annotation.createdAt IN LAST 90 DAYS ``` * **Explanation:** This query demonstrates the core use case. Instead of analyzing the project's state directly,`SHOW TREND` operates on the `ANNOTATION SPINE`. It gathers all annotations attached to "Project Chimera" that have a `classification_spine` of "ProjectRiskScore", extracts their `classification_value` (e.g., a score from 1-10), and calculates the statistical trend of those scores over the last 90 days. The result directly answers whether the perception of risk is increasing or decreasing. **Example 9: Measuring Consensus to Drive Decision-Making** * **Question:** "As we approach the launch date for 'Product X', is the team reaching a consensus on its market viability, or is there still significant disagreement?" * **NDL Query:** ```ndl SHOW CONSENSUS FOR (prod:Product {name: "Product X"}) ON ANNOTATION SPINE "MarketViability" CONTEXT (ann:Annotation)-[:ANNOTATES]->(prod) WHERE ann.createdBy IN ('senior_analyst_team') -- Filter for a specific user group ``` * **Explanation:** This query uses the `CONSENSUS` metric to measure the standard deviation of viability scores provided by senior analysts. A low result indicates the team is aligned, providing confidence for a go/no-go decision. A high result instantly flags that there are diverging opinions that must be addressed, preventing a premature decision based on a flawed assumption of agreement. The `CONTEXT` and `WHERE` clauses show how this can be combined with other filters for highly specific insights.