feat: improved special characters usage on randomized strings

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sylphian
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

PLAN.md

@@ -0,0 +1,77 @@
+# LotusGen: Development Session & Feature Implementation
+
+## 1. Environment & Operating Rules
+- **Workspace Directory**: Strictly confined to `C:\Users\admin\Documents\Code\lotusgen`. Never read, access, or write outside this directory.
+- **Python Environment**: Managed strictly via `uv`. Always use `backend/.venv/Scripts/python.exe` and `backend/.venv/Scripts/pytest.exe`. Never install to or modify global python site-packages.
+- **Git**: Never perform git operations autonomously. Leave all git commits, pushes, and branches to the user.
+- **Design System**: Strict editorial minimalism (per `SKILL.md`):
+  - Palette: `#000000`, `#FFFFFF`, `#F4FAFF`, `#DEE3E8`, `#ADBFCE`, `#4F6170`, `#525456`.
+  - Dark mode supported via `.dark` class.
+  - Borders: Crisp `1px` or `1.5px` borders (`#DEE3E8` / `#4F6170`, pure black/white on hover/focus).
+  - Icons: Exclusively `@phosphor-icons/react` (no Lucide, no Feather, no Heroicons).
+  - Zero heavy box shadows, zero gradients, zero emojis in UI code.
+- **Routing**: Lightweight native HTML5 History routing via `useRoute.ts`. All project and table routes use strict alphanumeric-and-hyphen slugs (`^[a-z0-9-]+$`):
+  - Homepage: `/`
+  - Project Workspace: `/:projectSlug`
+  - Table Editor: `/:projectSlug/:tableSlug`
+
+---
+
+## 2. Current Architecture & Codebase State
+- **Backend Stack**: FastAPI, SQLAlchemy async with `aiosqlite`, Polars and PyArrow streaming generators, `sse-starlette` real-time telemetry.
+- **Frontend Stack**: React 19, TypeScript, Vite, Tailwind CSS v4, `@phosphor-icons/react`.
+- **Table Archetypes**:
+  1. `record`: Fact/event streams with randomized, function, reference, and incremental columns.
+  2. `aggregate`: Dimensional BI marts with Cartesian loops and sublist/collection sampling.
+  3. `reference`: 2D dimension catalogs with interactive grid.
+  4. `collection`: 1D weighted lists with probability weights (uniform or custom weights).
+- **Key Mechanics Already Implemented**:
+  - Top-to-bottom intra-table DAG validation (`dag.py`).
+  - Inter-table acyclic DAG and dependency locking (`valid` tables referenced by others cannot be drafted, renamed, or deleted).
+  - Sandboxed Python formulas with native f-strings, string ops, and math.
+  - Active/muted column toggle: Inactive columns (`active: false`) are compute-only helpers; they are omitted from preview and CSV/Parquet output, but remain fully referenceable by upstream/downstream formulas.
+  - Auto-trimming: All table names, slugs, field configs, choices, and data cells are stripped of leading/trailing whitespace on save and validation.
+  - Backend test suite: 15 passing pytest tests in `backend/tests/`.
+
+---
+
+## 3. Scope of Work for this Session
+
+Implement the following 4 features:
+
+### Feature 1: Unified "Import / Export" Modal for All Tables
+Replace the standalone file-picker buttons with a single **"Import / Export"** button in `TableSchemaEditor.tsx`, `ReferenceSpreadsheet.tsx`, and `CollectionEditor.tsx`.
+- Opens a pop-up modal with switchable tabs stacked horizontally on the top-left:
+  - **Export Tab**: Displays a formatted, `readOnly` textarea of the table's JSON, with a **"Copy to Clipboard"** button (with 2-second "Copied!" feedback) and a **"Download .json"** button.
+  - **Import Tab**: Displays an editable `<textarea>` to paste JSON.
+  - **Confirmation Guard**: Clicking "Import" triggers a confirmation dialog:
+    > *"Are you sure you want to import? This will overwrite the current columns/data in this table."*
+    Proceeding parses the JSON, updates editor state, and shows a success toast.
+
+### Feature 2: Specialized Collection & Reference Import Behavior
+- **Collection Table (`CollectionEditor.tsx`)**:
+  - The modal has `[ Import ]` and `[ Export ]` tabs.
+  - **Remove the file browse input completely**.
+  - The Import tab's textarea accepts structured JSON **OR** plain text where **single-line = single-value** (each line becomes an item with `weight: 1.0`).
+- **Reference Table (`ReferenceSpreadsheet.tsx`)**:
+  - The modal has 3 tabs stacked horizontally on top-left: `[ Import JSON ]`, `[ CSV Input ]`, `[ Export JSON ]`.
+  - The **`[ CSV Input ]`** tab provides a file upload strictly for `.csv` files (`accept=".csv"`), retaining the existing Column Mismatch Warning modal.
+
+### Feature 3: Table Card Actions (Duplicate & Rename) in `ProjectDetail.tsx`
+- **Duplicate Table Button (`Copy` icon)**:
+  - Added to the bottom action row on each table card.
+  - Creates an exact duplicate of the table:
+    - Name: `<original_table>_copy` (auto-incrementing to `_copy_2` if taken).
+    - Slug: `<original_slug>-copy` (sanitized to `^[a-z0-9-]+$`).
+    - Copies archetype, format, row limit, exclude_if, fields, columns, and rows data.
+    - Duplicate starts in `draft` status.
+- **Rename Table Button (`PencilSimple` icon)**:
+  - Added next to the table name heading in the card header.
+  - Opens a modal to edit `Table Name` and `Table Slug` (validated against `^[a-z0-9-]+$`).
+  - If the table is `valid` and actively referenced by other tables, disable/block renaming with a clear lock notice explaining dependants would break.
+
+---
+
+## 4. Verification Requirements
+1. Run backend pytest tests (`$env:PYTHONPATH="."; backend\.venv\Scripts\pytest.exe -v backend\tests\`) and ensure all existing and new tests pass.
+2. Run frontend production build (`npm --prefix frontend run build`) and ensure TypeScript compilation and asset bundling succeed with zero errors.

README.md

@@ -0,0 +1,52 @@
+# LotusGen
+
+> High-throughput, low-memory dummy and synthetic tabular data generator.
+
+LotusGen is a lightweight, self-hosted web application designed to simplify the generation of complex, relational mock datasets. It was created to provide developers, QA engineers, and data practitioners with a fast, intuitive way to design relational schemas, preview distributions, and generate millions of deterministic rows in CSV and Apache Parquet formats.
+
+---
+
+### ⚠️ Intended Use & Disclaimer
+
+LotusGen is built strictly for development, local benchmarking, integration testing, and QA environments. 
+
+It is designed for generating fictitious, synthetic mock data and is not intended, tested, or recommended for creating, processing, or substituting real production data.
+
+---
+
+### Features at a Glance
+
+- Multi-Archetype Schemas: Model Record, Aggregate, Reference (lookup dimensions), and Collection (weighted choice pools) tables.
+- Relational Integrity: Intra-table DAG validation ensures clean topological evaluation of calculated fields and foreign references.
+- AST-Sandboxed Formulas: Safe row-by-row expressions without unsafe execution.
+- Streaming Generation: Powered by Polars and PyArrow for bounded memory usage during large-scale exports.
+- 100% Deterministic: Seed-based pseudorandom generation guarantees identical output across runs.
+
+---
+
+### Quick Start
+
+Run LotusGen locally using Docker:
+
+```bash
+Clone the repository
+git clone https://git.sylphian.my.id/Sylphian/lotusgen.git
+cd lotusgen
+
+Start the container
+docker compose up -d --build
+```
+
+Access the web interface at `http://localhost:8000`.
+
+---
+
+### Documentation & Wiki
+
+For full architecture details, table archetype guides, formula syntax, column configurations, and API references, please visit our [Project Wiki](https://git.sylphian.my.id/Sylphian/lotusgen/wiki).
+
+---
+
+### License
+
+This project is open-source software licensed under the [MIT License](LICENSE).

backend/api/routes.py

@@ -227,10 +227,20 @@ async def preview_table(table_id: str, count: int = Query(10, le=50), db: AsyncS
 
     project_tables = await TableService.list_tables_by_project(db, table.project_id)
     all_dicts = [model_to_table_dict(t) for t in project_tables]
-    ref_tables = [t for t in all_dicts if t.get("archetype") == "reference"]
+    ref_tables = [t for t in all_dicts if t.get("archetype") in ("reference", "collection")]
+
+    proj = await ProjectService.get_project(db, table.project_id)
+    proj_name = proj.name if proj else ""
+    proj_seed = proj.seed if proj else 42
 
     try:
-        samples = generate_dry_run_sample(model_to_table_dict(table), ref_tables, sample_size=count)
+        samples = generate_dry_run_sample(
+            model_to_table_dict(table),
+            ref_tables,
+            sample_size=count,
+            project_name=proj_name,
+            seed=proj_seed,
+        )
         return DryRunResponse(success=True, sample_rows=samples)
     except Exception as e:
         return DryRunResponse(success=False, error=f"{type(e).__name__}: {str(e)}")

backend/engine/generators.py

@@ -78,10 +78,48 @@ def build_collection_indices(collection_tables: List[Dict[str, Any]]) -> Dict[st
             else:
                 values.append(row)
                 weights.append(1.0)
-        indices[t_name] = {"values": values, "weights": weights}
+        entry = {"values": values, "weights": weights}
+        indices[t_name] = entry
+        if t.get("slug"):
+            indices[t["slug"]] = entry
+        if t.get("id"):
+            indices[t["id"]] = entry
     return indices
 
 
+def calculate_cartesian_count(table_def: Dict[str, Any], coll_indices: Dict[str, Any]) -> int:
+    fields = table_def.get("fields", [])
+    dim_fields = [f for f in fields if f.get("role") == "aggregated"]
+    if not dim_fields:
+        return max(1, table_def.get("row_limit") or 1)
+    total = 1
+    for df in dim_fields:
+        cfg = df.get("config") or {}
+        col_type = df.get("type", "string")
+        mode = cfg.get("mode", "all")
+        if col_type == "datetime":
+            days = int(cfg.get("days", 30))
+            total *= max(1, days)
+        elif col_type == "collection" or "collection_table" in cfg:
+            coll_name = cfg.get("collection_table")
+            coll_data = coll_indices.get(coll_name, {})
+            vals = coll_data.get("values", [])
+            default_len = len(vals) if vals else 1
+            min_len = int(cfg.get("min_len", default_len))
+            total *= max(1, min_len)
+        else:
+            vals = cfg.get("choices", [])
+            if mode == "sublist":
+                default_len = len(vals) if vals else 1
+                min_len = int(cfg.get("min_len", default_len))
+                total *= max(1, min_len)
+            else:
+                total *= max(1, len(vals))
+    if table_def.get("row_limit"):
+        return max(1, min(table_def["row_limit"], total))
+    return max(1, total)
+
+
 class TableGenerator:
     def __init__(
         self,
@@ -89,12 +127,17 @@ class TableGenerator:
         ref_tables: List[Dict[str, Any]],
         seed: Optional[int] = 42,
         chunk_size: int = 50000,
+        project_name: Optional[str] = None,
+        total_count: Optional[int] = None,
     ):
         self.table = table
         self.ref_tables = ref_tables
-        self.seed = seed
+        self.seed = seed if seed is not None else 42
         self.chunk_size = chunk_size
-        self.rng = random.Random(seed)
+        self.rng = random.Random(self.seed)
+        self.table_name = table.get("name", "")
+        self.project_name = project_name or table.get("project_name", "")
+        self.timestamp = datetime.datetime.now(datetime.timezone.utc).isoformat()
         self.ref_indices = build_reference_indices(
             [t for t in ref_tables if t.get("archetype") == "reference"]
         )
@@ -102,6 +145,13 @@ class TableGenerator:
             [t for t in ref_tables if t.get("archetype") == "collection"]
         )
 
+        if total_count is not None:
+            self.total_count = max(1, total_count)
+        elif table.get("archetype") == "aggregate":
+            self.total_count = calculate_cartesian_count(table, self.collection_indices)
+        else:
+            self.total_count = max(1, table.get("row_limit") or 100000)
+
         # Pre-compile formulas
         self.fields = table.get("fields", [])
         self.output_fields = [f["name"] for f in self.fields if f.get("active", True)]
@@ -119,10 +169,15 @@ class TableGenerator:
         if exclude_if and exclude_if.strip():
             self.compiled_exclude = CompiledExpression(exclude_if, target_name="exclude_if")
 
-    def _sample_randomized(self, field: Dict[str, Any]) -> Any:
+    def _sample_randomized(self, field: Dict[str, Any], row_idx: int = 0) -> Any:
         col_type = field.get("type", "string")
         cfg = field.get("config") or {}
 
+        if col_type == "incremental":
+            start = int(cfg.get("start", 1))
+            step = int(cfg.get("step", 1))
+            return start + row_idx * step
+
         if col_type == "collection" or "collection_table" in cfg:
             coll_name = cfg.get("collection_table")
             coll_data = self.collection_indices.get(coll_name, {})
@@ -223,7 +278,15 @@ class TableGenerator:
 
             for row_idx in range(batch_count):
                 abs_row_idx = rows_emitted + row_idx
-                row_dict: Dict[str, Any] = {}
+                row_dict: Dict[str, Any] = {
+                    "_index": abs_row_idx,
+                    "_row": abs_row_idx + 1,
+                    "_count": self.total_count,
+                    "_name": self.table_name,
+                    "_project": self.project_name,
+                    "_timestamp": self.timestamp,
+                    "_seed": self.seed,
+                }
 
                 for f in self.fields:
                     name = f["name"]
@@ -233,10 +296,15 @@ class TableGenerator:
                     if role == "static":
                         val = (f.get("config") or {}).get("value")
                     elif role == "randomized":
-                        if name in datetime_pools:
+                        if col_type == "incremental":
+                            cfg = f.get("config") or {}
+                            start = int(cfg.get("start", 1))
+                            step = int(cfg.get("step", 1))
+                            val = start + abs_row_idx * step
+                        elif name in datetime_pools:
                             val = datetime_pools[name][abs_row_idx]
                         else:
-                            val = self._sample_randomized(f)
+                            val = self._sample_randomized(f, row_idx=abs_row_idx)
                     elif role == "reference":
                         val = self._lookup_reference(f, row_dict)
                     elif role == "function":
@@ -322,6 +390,13 @@ class TableGenerator:
             if dim_idx == len(dim_descriptors):
                 # Innermost level: evaluate exclude_if, reference, randomized, function
                 row_copy = dict(current_row)
+                row_copy["_index"] = total_generated
+                row_copy["_row"] = total_generated + 1
+                row_copy["_count"] = self.total_count
+                row_copy["_name"] = self.table_name
+                row_copy["_project"] = self.project_name
+                row_copy["_timestamp"] = self.timestamp
+                row_copy["_seed"] = self.seed
 
                 if self.compiled_exclude:
                     # Check if combination should be excluded
@@ -336,7 +411,13 @@ class TableGenerator:
                     if role == "static":
                         val = (f.get("config") or {}).get("value")
                     elif role == "randomized":
-                        val = self._sample_randomized(f)
+                        if col_type == "incremental":
+                            cfg = f.get("config") or {}
+                            start = int(cfg.get("start", 1))
+                            step = int(cfg.get("step", 1))
+                            val = start + total_generated * step
+                        else:
+                            val = self._sample_randomized(f, row_idx=total_generated)
                     elif role == "reference":
                         val = self._lookup_reference(f, row_copy)
                     elif role == "function":
@@ -348,8 +429,8 @@ class TableGenerator:
                     val = cast_type(val, col_type)
                     row_copy[name] = val
 
-                for k, v in row_copy.items():
+                for f in self.fields:
-                    buffer[k].append(v)
+                    buffer[f["name"]].append(row_copy[f["name"]])
 
                 total_generated += 1
                 rows_emitted += 1
@@ -408,7 +489,11 @@ def generate_reference_table_df(table: Dict[str, Any]) -> pl.DataFrame:
 
 
 def generate_dry_run_sample(
-    table: Dict[str, Any], ref_tables: List[Dict[str, Any]], sample_size: int = 10
+    table: Dict[str, Any],
+    ref_tables: List[Dict[str, Any]],
+    sample_size: int = 10,
+    project_name: Optional[str] = None,
+    seed: Optional[int] = 123,
 ) -> List[Dict[str, Any]]:
     validate_intra_table_dag(table)
     archetype = table.get("archetype", "record")
@@ -417,7 +502,22 @@ def generate_dry_run_sample(
         df = generate_reference_table_df(table)
         return df.head(sample_size).to_dicts()
 
-    gen = TableGenerator(table, ref_tables, seed=123, chunk_size=sample_size)
+    coll_indices = build_collection_indices(
+        [t for t in ref_tables if t.get("archetype") == "collection"]
+    )
+    if archetype == "record":
+        target_count = max(1, table.get("row_limit") or 100000)
+    else:
+        target_count = calculate_cartesian_count(table, coll_indices)
+
+    gen = TableGenerator(
+        table,
+        ref_tables,
+        seed=seed if seed is not None else 123,
+        chunk_size=sample_size,
+        project_name=project_name,
+        total_count=target_count,
+    )
     if archetype == "record":
         chunks = list(gen.generate_record_chunks(total_rows=sample_size))
     else:

backend/engine/samplers.py

@@ -77,11 +77,21 @@ def sample_weighted_without_replacement(
     return [item for _, item in keys[:chosen_size]]
 
 
+HEX_UPPERCASE = "0123456789ABCDEF"
+
+
 def sample_regex_mask(mask: str, rng: random.Random) -> str:
-    # '#' -> digit, '?' -> uppercase, '@' -> lowercase, '*' -> alphanumeric
+    # '#' -> digit, '?' -> uppercase, '@' -> lowercase, '*' -> alphanumeric, '%' -> hex uppercase
+    # '\' -> escape next character literally
     out = []
+    escape = False
     for char in mask:
-        if char == "#":
+        if escape:
+            out.append(char)
+            escape = False
+        elif char == "\\":
+            escape = True
+        elif char == "#":
             out.append(rng.choice(string.digits))
         elif char == "?":
             out.append(rng.choice(string.ascii_uppercase))
@@ -89,8 +99,12 @@ def sample_regex_mask(mask: str, rng: random.Random) -> str:
             out.append(rng.choice(string.ascii_lowercase))
         elif char == "*":
             out.append(rng.choice(string.ascii_letters + string.digits))
+        elif char == "%":
+            out.append(rng.choice(HEX_UPPERCASE))
         else:
             out.append(char)
+    if escape:
+        out.append("\\")
     return "".join(out)
 
 
@@ -98,7 +112,7 @@ def cast_type(value: Any, col_type: str) -> Any:
     if value is None:
         return None
     try:
-        if col_type == "integer":
+        if col_type in ("integer", "incremental"):
             return int(value)
         elif col_type == "float":
             return float(value)

backend/engine/sandbox.py

@@ -52,6 +52,17 @@ class FormulaRuntimeError(Exception):
     pass
 
 
+BUILTIN_ROW_VARS = {
+    "_index",
+    "_row",
+    "_count",
+    "_name",
+    "_project",
+    "_timestamp",
+    "_seed",
+}
+
+
 def extract_variable_names(expression_str: str) -> Set[str]:
     if not expression_str or not expression_str.strip():
         return set()
@@ -60,7 +71,7 @@ def extract_variable_names(expression_str: str) -> Set[str]:
     except SyntaxError as e:
         raise FormulaSyntaxError(f"Syntax error in expression '{expression_str}': {e.msg}") from e
 
-    reserved = set(SAFE_GLOBALS.keys()) | set(SAFE_BUILTINS.keys())
+    reserved = set(SAFE_GLOBALS.keys()) | set(SAFE_BUILTINS.keys()) | BUILTIN_ROW_VARS
     vars_found: Set[str] = set()
     for node in ast.walk(tree):
         if isinstance(node, ast.Name):

backend/schemas/models.py

@@ -18,7 +18,7 @@ def validate_slug_format(slug: Optional[str]) -> Optional[str]:
 class FieldSchema(BaseModel):
     name: str
     role: Literal["aggregated", "randomized", "function", "reference", "static"]
-    type: Literal["string", "integer", "float", "datetime", "boolean", "uuid7", "choice", "collection"]
+    type: Literal["string", "integer", "float", "datetime", "boolean", "uuid7", "choice", "collection", "incremental"]
     active: bool = True
     config: Optional[Dict[str, Any]] = Field(default_factory=dict)
 

backend/services/generation_service.py

@@ -13,6 +13,7 @@ from backend.engine.generators import (
     generate_reference_table_df,
 )
 from backend.engine.writers import StreamingCsvWriter, StreamingParquetWriter
+from backend.services.project_service import ProjectService
 from backend.services.table_service import TableService, model_to_table_dict
 
 
@@ -110,6 +111,10 @@ class GenerationService:
             table_dict = model_to_table_dict(table)
             ref_tables = [t for t in all_dicts if t.get("archetype") in ("reference", "collection")]
 
+            project = await ProjectService.get_project(session, project_id)
+            project_name = project.name if project else ""
+            project_seed = project.seed if project else 42
+
         GenerationService._broadcast(
             job_id,
             {
@@ -137,7 +142,14 @@ class GenerationService:
                 writer.write_chunk(df)
                 total_written = len(df)
             elif archetype == "record":
-                gen = TableGenerator(table_dict, ref_tables, seed=42, chunk_size=50000)
+                gen = TableGenerator(
+                    table_dict,
+                    ref_tables,
+                    seed=project_seed,
+                    chunk_size=50000,
+                    project_name=project_name,
+                    total_count=target_rows,
+                )
                 for chunk in gen.generate_record_chunks(total_rows=target_rows):
                     writer.write_chunk(chunk)
                     total_written += len(chunk)
@@ -159,7 +171,14 @@ class GenerationService:
                     await asyncio.sleep(0.001)  # Yield to event loop
             else:
                 # aggregate
-                gen = TableGenerator(table_dict, ref_tables, seed=42, chunk_size=50000)
+                gen = TableGenerator(
+                    table_dict,
+                    ref_tables,
+                    seed=project_seed,
+                    chunk_size=50000,
+                    project_name=project_name,
+                    total_count=target_rows,
+                )
                 for chunk in gen.generate_aggregate_chunks():
                     writer.write_chunk(chunk)
                     total_written += len(chunk)

backend/services/table_service.py

@@ -3,7 +3,7 @@ import uuid
 from typing import Any, Dict, List, Optional
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
-from backend.db.models import TableModel
+from backend.db.models import ProjectModel, TableModel
 from backend.engine.dag import (
     DAGValidationError,
     get_referencing_tables,
@@ -22,6 +22,64 @@ class TableValidationError(Exception):
     pass
 
 
+def sanitize_value(v: Any) -> Any:
+    if isinstance(v, str):
+        return v.strip()
+    return v
+
+
+def sanitize_config(cfg: Optional[Dict[str, Any]]) -> Dict[str, Any]:
+    if not cfg:
+        return {}
+    res = {}
+    for k, v in cfg.items():
+        if isinstance(v, str):
+            res[k] = v.strip()
+        elif isinstance(v, list):
+            res[k] = [sanitize_value(item) for item in v]
+        elif isinstance(v, dict):
+            res[k] = {sanitize_value(dk): sanitize_value(dv) for dk, dv in v.items()}
+        else:
+            res[k] = v
+    return res
+
+
+def sanitize_fields_data(fields: Optional[List[Any]]) -> List[Dict[str, Any]]:
+    if not fields:
+        return []
+    res = []
+    for f in fields:
+        f_dict = f.model_dump() if hasattr(f, "model_dump") else dict(f)
+        if "name" in f_dict and isinstance(f_dict["name"], str):
+            f_dict["name"] = f_dict["name"].strip()
+        if "config" in f_dict:
+            f_dict["config"] = sanitize_config(f_dict.get("config"))
+        res.append(f_dict)
+    return res
+
+
+def sanitize_columns_data(cols: Optional[List[Any]]) -> List[str]:
+    if not cols:
+        return []
+    return [str(c).strip() for c in cols if c is not None]
+
+
+def sanitize_rows_data(rows: Optional[List[Any]]) -> List[Any]:
+    if not rows:
+        return []
+    res = []
+    for row in rows:
+        if isinstance(row, (list, tuple)):
+            res.append([sanitize_value(cell) for cell in row])
+        elif isinstance(row, dict):
+            res.append({sanitize_value(k): sanitize_value(v) for k, v in row.items()})
+        elif isinstance(row, str):
+            res.append(row.strip())
+        else:
+            res.append(row)
+    return res
+
+
 def model_to_table_dict(t: TableModel) -> Dict[str, Any]:
     return {
         "id": t.id,
@@ -41,13 +99,13 @@ def model_to_table_dict(t: TableModel) -> Dict[str, Any]:
 class TableService:
     @staticmethod
     async def create_table(session: AsyncSession, project_id: str, data: TableCreate) -> TableModel:
-        # Check for unique table name within project
+        clean_name = data.name.strip()
-        stmt = select(TableModel).where(TableModel.project_id == project_id, TableModel.name == data.name)
+        stmt = select(TableModel).where(TableModel.project_id == project_id, TableModel.name == clean_name)
         existing = (await session.execute(stmt)).scalar_one_or_none()
         if existing:
-            raise TableValidationError(f"A table named '{data.name}' already exists in this project.")
+            raise TableValidationError(f"A table named '{clean_name}' already exists in this project.")
 
-        base_slug = data.slug or re.sub(r"[^a-z0-9-]+", "-", data.name.lower()).strip("-") or "table"
+        base_slug = (data.slug.strip() if data.slug else None) or re.sub(r"[^a-z0-9-]+", "-", clean_name.lower()).strip("-") or "table"
         slug = base_slug
         counter = 1
         while True:
@@ -58,20 +116,25 @@ class TableService:
             slug = f"{base_slug}-{counter}"
             counter += 1
 
+        clean_fields = sanitize_fields_data(data.fields)
+        clean_columns = sanitize_columns_data(data.columns)
+        clean_rows = sanitize_rows_data(data.data)
+        clean_exclude = data.exclude_if.strip() if data.exclude_if is not None else None
+
         table_id = f"tbl_{uuid.uuid4().hex[:8]}"
         table = TableModel(
             id=table_id,
             project_id=project_id,
             slug=slug,
-            name=data.name,
+            name=clean_name,
             archetype=data.archetype,
             status="draft",
             format=data.format or "csv",
             row_limit=data.row_limit,
-            exclude_if=data.exclude_if,
+            exclude_if=clean_exclude,
-            fields_data=[f.model_dump() for f in (data.fields or [])],
+            fields_data=clean_fields,
-            columns_data=data.columns or [],
+            columns_data=clean_columns,
-            rows_data=data.data or [],
+            rows_data=clean_rows,
         )
         session.add(table)
         await session.commit()
@@ -113,9 +176,10 @@ class TableService:
         all_dicts = [model_to_table_dict(t) for t in project_tables]
         referencers = get_referencing_tables(table.name, all_dicts)
 
+        clean_name = data.name.strip() if data.name is not None else None
         if referencers and table.status == "valid":
             # If changing name or schema while referenced, block it!
-            if data.name is not None and data.name != table.name:
+            if clean_name is not None and clean_name != table.name:
                 raise TableLockedError(
                     f"Cannot rename table '{table.name}': actively referenced by {referencers}"
                 )
@@ -124,25 +188,25 @@ class TableService:
                     f"Cannot alter schema of valid table '{table.name}': actively referenced by {referencers}. Invalidate or detach downstream tables first."
                 )
 
-        if data.name is not None:
+        if clean_name is not None:
-            table.name = data.name
+            table.name = clean_name
         if data.slug is not None:
-            table.slug = data.slug
+            table.slug = data.slug.strip()
         if data.format is not None:
             table.format = data.format
         if data.row_limit is not None:
             table.row_limit = data.row_limit
         if data.exclude_if is not None:
-            table.exclude_if = data.exclude_if
+            table.exclude_if = data.exclude_if.strip()
         if data.fields is not None:
-            table.fields_data = [f.model_dump() for f in data.fields]
+            table.fields_data = sanitize_fields_data(data.fields)
             # Modifying draft table schema automatically keeps it draft
             table.status = "draft"
         if data.columns is not None:
-            table.columns_data = data.columns
+            table.columns_data = sanitize_columns_data(data.columns)
             table.status = "draft"
         if data.data is not None:
-            table.rows_data = data.data
+            table.rows_data = sanitize_rows_data(data.data)
 
         await session.commit()
         await session.refresh(table)
@@ -173,6 +237,21 @@ class TableService:
         if not table:
             raise ValueError("Table not found")
 
+        if table.name:
+            table.name = table.name.strip()
+        if table.slug:
+            table.slug = table.slug.strip()
+        if table.exclude_if:
+            table.exclude_if = table.exclude_if.strip()
+        if table.fields_data:
+            table.fields_data = sanitize_fields_data(table.fields_data)
+        if table.columns_data:
+            table.columns_data = sanitize_columns_data(table.columns_data)
+        if table.rows_data:
+            table.rows_data = sanitize_rows_data(table.rows_data)
+        await session.commit()
+        await session.refresh(table)
+
         project_tables = await TableService.list_tables_by_project(session, table.project_id)
         all_dicts = [model_to_table_dict(t) for t in project_tables]
         current_dict = model_to_table_dict(table)
@@ -209,8 +288,19 @@ class TableService:
                 raise TableValidationError(str(e)) from e
 
         # 3. 10-Row Dry-Run Generation
+        proj_res = await session.execute(select(ProjectModel).where(ProjectModel.id == table.project_id))
+        project = proj_res.scalar_one_or_none()
+        project_name = project.name if project else ""
+        project_seed = project.seed if project else 42
+
         try:
-            sample_rows = generate_dry_run_sample(current_dict, ref_tables, sample_size=10)
+            sample_rows = generate_dry_run_sample(
+                current_dict,
+                ref_tables,
+                sample_size=10,
+                project_name=project_name,
+                seed=project_seed,
+            )
         except Exception as e:
             raise TableValidationError(f"Dry-run generation failed: {type(e).__name__} - {e}") from e
 

backend/tests/test_api.py

@@ -331,6 +331,15 @@ async def test_collection_table_api_workflow():
         assert res_inv.status_code == 409
         assert "actively referenced" in res_inv.json()["detail"]
 
+        # Test preview endpoint for table with collection column
+        res_prev = await client.get(f"/api/tables/{sales_id}/preview?count=5")
+        assert res_prev.status_code == 200
+        prev_data = res_prev.json()
+        assert prev_data["success"] is True
+        assert len(prev_data["sample_rows"]) == 5
+        for row in prev_data["sample_rows"]:
+            assert row["pay_type"] in ("Cash", "Card")
+
         # 6. Generate sales
         res_gen = await client.post(f"/api/projects/{proj_slug}/tables/{sales_id}/generate")
         assert res_gen.status_code == 200
@@ -346,3 +355,252 @@ async def test_collection_table_api_workflow():
         res_final = await client.get(f"/api/jobs/{job_id}")
         assert res_final.json()["status"] == "completed"
         assert res_final.json()["rows_generated"] == 20
+
+
+@pytest.mark.asyncio
+async def test_whitespace_sanitization_on_save_and_validate():
+    await init_db()
+    transport = ASGITransport(app=app)
+    async with AsyncClient(transport=transport, base_url="http://test") as client:
+        proj_slug = f"trim-test-{uuid.uuid4().hex[:6]}"
+        res_p = await client.post(
+            "/api/projects",
+            json={"name": "Trim Project", "slug": proj_slug},
+        )
+        assert res_p.status_code == 201
+
+        res_ref = await client.post(
+            f"/api/projects/{proj_slug}/tables",
+            json={
+                "name": "  ref_methods  ",
+                "slug": "  ref-methods  ",
+                "archetype": "reference",
+                "columns": ["  code  ", "  label  "],
+                "data": [["  M01  ", "  Credit Card  "], ["  M02  ", "  Debit  "]],
+            },
+        )
+        assert res_ref.status_code == 201
+        ref_data = res_ref.json()
+        assert ref_data["name"] == "ref_methods"
+        assert ref_data["slug"] == "ref-methods"
+        assert ref_data["columns"] == ["code", "label"]
+        assert ref_data["data"] == [["M01", "Credit Card"], ["M02", "Debit"]]
+
+        val_ref = await client.post(f"/api/tables/{ref_data['id']}/validate")
+        assert val_ref.status_code == 200
+        assert val_ref.json()["success"] is True
+
+        res_rec = await client.post(
+            f"/api/projects/{proj_slug}/tables",
+            json={
+                "name": "  transactions  ",
+                "slug": "  transactions  ",
+                "archetype": "record",
+                "exclude_if": "  amount < 0  ",
+                "fields": [
+                    {
+                        "name": "  pay_code  ",
+                        "role": "randomized",
+                        "type": "choice",
+                        "config": {"choices": ["  M01  ", "  M02  "], "choices_text": "  M01, M02  "},
+                    },
+                    {
+                        "name": "  pay_label  ",
+                        "role": "reference",
+                        "type": "string",
+                        "config": {
+                            "source_field": "  pay_code  ",
+                            "ref_table": "  ref_methods  ",
+                            "key_col": "  code  ",
+                            "val_col": "  label  ",
+                        },
+                    },
+                    {
+                        "name": "  amount  ",
+                        "role": "static",
+                        "type": "string",
+                        "config": {"value": "  100  "},
+                    },
+                    {
+                        "name": "  calc  ",
+                        "role": "function",
+                        "type": "string",
+                        "config": {"expression": "  f'{pay_label}_{amount}'  "},
+                    },
+                ],
+            },
+        )
+        assert res_rec.status_code == 201
+        rec_data = res_rec.json()
+        assert rec_data["name"] == "transactions"
+        assert rec_data["slug"] == "transactions"
+        assert rec_data["exclude_if"] == "amount < 0"
+        fields = rec_data["fields"]
+        assert fields[0]["name"] == "pay_code"
+        assert fields[0]["config"]["choices"] == ["M01", "M02"]
+        assert fields[0]["config"]["choices_text"] == "M01, M02"
+        assert fields[1]["name"] == "pay_label"
+        assert fields[1]["config"]["source_field"] == "pay_code"
+        assert fields[1]["config"]["ref_table"] == "ref_methods"
+        assert fields[1]["config"]["key_col"] == "code"
+        assert fields[1]["config"]["val_col"] == "label"
+        assert fields[2]["name"] == "amount"
+        assert fields[2]["config"]["value"] == "100"
+        assert fields[3]["name"] == "calc"
+        assert fields[3]["config"]["expression"] == "f'{pay_label}_{amount}'"
+
+        val_rec = await client.post(f"/api/tables/{rec_data['id']}/validate")
+        assert val_rec.status_code == 200
+        assert val_rec.json()["success"] is True
+
+
+@pytest.mark.asyncio
+async def test_table_rename_and_duplicate_workflow():
+    await init_db()
+    transport = ASGITransport(app=app)
+    async with AsyncClient(transport=transport, base_url="http://test") as client:
+        # 1. Create Project
+        proj_slug = f"dup-test-{uuid.uuid4().hex[:6]}"
+        res_p = await client.post(
+            "/api/projects",
+            json={"name": "Duplicate Test Project", "slug": proj_slug},
+        )
+        assert res_p.status_code == 201
+
+        # 2. Create Reference Table
+        res_ref = await client.post(
+            f"/api/projects/{proj_slug}/tables",
+            json={
+                "name": "ref_types",
+                "slug": "ref-types",
+                "archetype": "reference",
+                "columns": ["id", "label"],
+                "data": [["1", "Type A"], ["2", "Type B"]],
+            },
+        )
+        assert res_ref.status_code == 201
+        ref_id = res_ref.json()["id"]
+
+        # Validate ref table
+        val_ref = await client.post(f"/api/tables/{ref_id}/validate")
+        assert val_ref.json()["success"] is True
+
+        # 3. Create Record Table referencing ref_types
+        res_rec = await client.post(
+            f"/api/projects/{proj_slug}/tables",
+            json={
+                "name": "records",
+                "slug": "records",
+                "archetype": "record",
+                "fields": [
+                    {"name": "id", "role": "randomized", "type": "uuid7"},
+                    {
+                        "name": "type_label",
+                        "role": "reference",
+                        "type": "string",
+                        "config": {
+                            "source_field": "id",
+                            "ref_table": "ref_types",
+                            "key_col": "id",
+                            "val_col": "label",
+                        },
+                    },
+                ],
+            },
+        )
+        assert res_rec.status_code == 201
+        rec_id = res_rec.json()["id"]
+
+        # Validate records table -> now ref_types is actively referenced by valid table!
+        val_rec = await client.post(f"/api/tables/{rec_id}/validate")
+        assert val_rec.json()["success"] is True
+
+        # 4. Attempt to rename ref_types while locked -> MUST fail with 409
+        res_bad_rename = await client.put(
+            f"/api/tables/{ref_id}",
+            json={"name": "ref_types_renamed"},
+        )
+        assert res_bad_rename.status_code == 409
+        assert "Cannot rename table" in res_bad_rename.json()["detail"]
+
+        # 5. Rename records table (which is not referenced by anything) -> succeeds!
+        res_rename = await client.put(
+            f"/api/tables/{rec_id}",
+            json={"name": "records_v2", "slug": "records-v2"},
+        )
+        assert res_rename.status_code == 200
+        assert res_rename.json()["name"] == "records_v2"
+        assert res_rename.json()["slug"] == "records-v2"
+
+        # 6. Duplicate table creation via POST /projects/{project_id}/tables
+        res_dup = await client.post(
+            f"/api/projects/{proj_slug}/tables",
+            json={
+                "name": "records_v2_copy",
+                "slug": "records-v2-copy",
+                "archetype": "record",
+                "fields": res_rename.json()["fields"],
+            },
+        )
+        assert res_dup.status_code == 201
+        dup_data = res_dup.json()
+        assert dup_data["name"] == "records_v2_copy"
+        assert dup_data["slug"] == "records-v2-copy"
+        assert dup_data["status"] == "draft"
+        assert len(dup_data["fields"]) == 2
+
+
+@pytest.mark.asyncio
+async def test_metadata_variables_api_workflow():
+    await init_db()
+    transport = ASGITransport(app=app)
+    async with AsyncClient(transport=transport, base_url="http://test") as client:
+        # 1. Create Project
+        proj_slug = f"telemetry-{uuid.uuid4().hex[:6]}"
+        res_p = await client.post(
+            "/api/projects",
+            json={"name": "TelemetryHub", "slug": proj_slug, "seed": 777},
+        )
+        assert res_p.status_code == 201
+
+        # 2. Create Table using _project, _name, _row, _count, _seed
+        res_tbl = await client.post(
+            f"/api/projects/{proj_slug}/tables",
+            json={
+                "name": "metrics",
+                "slug": "metrics",
+                "archetype": "record",
+                "row_limit": 2500,
+                "fields": [
+                    {"name": "id", "role": "randomized", "type": "incremental", "config": {"start": 1, "step": 1}},
+                    {
+                        "name": "tag",
+                        "role": "function",
+                        "type": "string",
+                        "config": {"expression": "f'{_project}:{_name}:{_row}:{_count}:{_seed}'"},
+                    },
+                ],
+            },
+        )
+        assert res_tbl.status_code == 201
+        tbl_id = res_tbl.json()["id"]
+
+        # 3. Call Preview Endpoint -> count=3, but _count reflects row_limit (2500)
+        res_prev = await client.get(f"/api/tables/{tbl_id}/preview?count=3")
+        assert res_prev.status_code == 200
+        prev_data = res_prev.json()
+        assert prev_data["success"] is True
+        rows = prev_data["sample_rows"]
+        assert len(rows) == 3
+        assert rows[0]["tag"] == "TelemetryHub:metrics:1:2500:777"
+        assert rows[1]["tag"] == "TelemetryHub:metrics:2:2500:777"
+        assert rows[2]["tag"] == "TelemetryHub:metrics:3:2500:777"
+
+        # 4. Validate Table
+        res_val = await client.post(f"/api/tables/{tbl_id}/validate")
+        assert res_val.status_code == 200
+        assert res_val.json()["success"] is True
+        val_rows = res_val.json()["sample_rows"]
+        assert len(val_rows) == 10
+        assert val_rows[0]["tag"] == "TelemetryHub:metrics:1:2500:777"
+        assert val_rows[9]["tag"] == "TelemetryHub:metrics:10:2500:777"

backend/tests/test_engine.py

@@ -19,7 +19,7 @@ from backend.engine.sandbox import (
     FormulaSyntaxError,
     extract_variable_names,
 )
-from backend.engine.samplers import generate_uuid7, get_random_sublist
+from backend.engine.samplers import generate_uuid7, get_random_sublist, sample_regex_mask
 from backend.engine.writers import StreamingCsvWriter, StreamingParquetWriter
 
 
@@ -463,3 +463,256 @@ def test_inactive_column_omitted_from_output_but_referenced_upstream():
     # Downstream formula resolved correctly using the inactive helper variables!
     for val in df["final_price"].to_list():
         assert val == 115.0
+
+
+def test_incremental_column_and_builtin_row_vars():
+    table = {
+        "name": "seq_test",
+        "archetype": "record",
+        "status": "valid",
+        "fields": [
+            {
+                "name": "row_id",
+                "role": "randomized",
+                "type": "incremental",
+                "config": {"start": 100, "step": 5},
+                "active": True,
+            },
+            {
+                "name": "code_from_row",
+                "role": "function",
+                "type": "string",
+                "config": {"expression": "f'ITEM-{_row:04d}_{_index}'"},
+                "active": True,
+            },
+            {
+                "name": "doubled_id",
+                "role": "function",
+                "type": "integer",
+                "config": {"expression": "row_id * 2"},
+                "active": True,
+            },
+        ],
+    }
+
+    validate_intra_table_dag(table)
+
+    gen = TableGenerator(table, [], seed=42, chunk_size=5)
+    chunks = list(gen.generate_record_chunks(total_rows=5))
+    df = pl.concat(chunks)
+
+    assert df["row_id"].to_list() == [100, 105, 110, 115, 120]
+    assert df["code_from_row"].to_list() == [
+        "ITEM-0001_0",
+        "ITEM-0002_1",
+        "ITEM-0003_2",
+        "ITEM-0004_3",
+        "ITEM-0005_4",
+    ]
+    assert df["doubled_id"].to_list() == [200, 210, 220, 230, 240]
+
+    agg_table = {
+        "name": "seq_agg",
+        "archetype": "aggregate",
+        "status": "valid",
+        "row_limit": 3,
+        "fields": [
+            {
+                "name": "dim",
+                "role": "aggregated",
+                "type": "choice",
+                "config": {"choices": ["A", "B", "C"], "mode": "all"},
+                "active": True,
+            },
+            {
+                "name": "seq_no",
+                "role": "randomized",
+                "type": "incremental",
+                "config": {"start": 1, "step": 1},
+                "active": True,
+            },
+            {
+                "name": "row_label",
+                "role": "function",
+                "type": "string",
+                "config": {"expression": "f'{dim}_{_row}'"},
+                "active": True,
+            },
+        ],
+    }
+
+    validate_intra_table_dag(agg_table)
+
+    gen_agg = TableGenerator(agg_table, [], seed=42, chunk_size=5)
+    chunks_agg = list(gen_agg.generate_aggregate_chunks())
+    df_agg = pl.concat(chunks_agg)
+
+    assert df_agg["seq_no"].to_list() == [1, 2, 3]
+    assert df_agg["row_label"].to_list() == ["A_1", "B_2", "C_3"]
+
+
+def test_extended_builtin_metadata_variables():
+    table = {
+        "name": "metadata_test",
+        "archetype": "record",
+        "status": "valid",
+        "row_limit": 5000,
+        "fields": [
+            {
+                "name": "id",
+                "role": "randomized",
+                "type": "incremental",
+                "config": {"start": 1, "step": 1},
+                "active": True,
+            },
+            {
+                "name": "summary",
+                "role": "function",
+                "type": "string",
+                "config": {"expression": "f'{_project}:{_name} row {_row}/{_count} (seed={_seed})'"},
+                "active": True,
+            },
+            {
+                "name": "iso_time",
+                "role": "function",
+                "type": "string",
+                "config": {"expression": "_timestamp"},
+                "active": True,
+            },
+        ],
+    }
+
+    validate_intra_table_dag(table)
+
+    gen = TableGenerator(
+        table,
+        [],
+        seed=101,
+        chunk_size=3,
+        project_name="WarehouseBI",
+        total_count=5000,
+    )
+    chunks = list(gen.generate_record_chunks(total_rows=3))
+    df = pl.concat(chunks)
+
+    assert df["summary"].to_list() == [
+        "WarehouseBI:metadata_test row 1/5000 (seed=101)",
+        "WarehouseBI:metadata_test row 2/5000 (seed=101)",
+        "WarehouseBI:metadata_test row 3/5000 (seed=101)",
+    ]
+    assert len(df["iso_time"][0]) >= 19
+
+    # Dry-run sample (preview) test: sample_size=2, but _count reflects configured row_limit (5000)!
+    samples = generate_dry_run_sample(
+        table,
+        [],
+        sample_size=2,
+        project_name="WarehouseBI",
+        seed=101,
+    )
+    assert len(samples) == 2
+    assert samples[0]["summary"] == "WarehouseBI:metadata_test row 1/5000 (seed=101)"
+    assert samples[1]["summary"] == "WarehouseBI:metadata_test row 2/5000 (seed=101)"
+
+    # Aggregate table cartesian count test
+    agg_table = {
+        "name": "agg_combos",
+        "archetype": "aggregate",
+        "status": "valid",
+        # row_limit is None -> cartesian product is 2 * 3 = 6
+        "row_limit": None,
+        "fields": [
+            {
+                "name": "cat",
+                "role": "aggregated",
+                "type": "choice",
+                "config": {"choices": ["X", "Y"], "mode": "all"},
+                "active": True,
+            },
+            {
+                "name": "sub",
+                "role": "aggregated",
+                "type": "choice",
+                "config": {"choices": ["1", "2", "3"], "mode": "all"},
+                "active": True,
+            },
+            {
+                "name": "meta_calc",
+                "role": "function",
+                "type": "string",
+                "config": {"expression": "f'{_row}/{_count}'"},
+                "active": True,
+            },
+        ],
+    }
+
+    validate_intra_table_dag(agg_table)
+    gen_agg = TableGenerator(
+        agg_table,
+        [],
+        seed=42,
+        project_name="CartesianProj",
+    )
+    chunks_agg = list(gen_agg.generate_aggregate_chunks())
+    df_agg = pl.concat(chunks_agg)
+    assert len(df_agg) == 6
+    assert df_agg["meta_calc"].to_list() == ["1/6", "2/6", "3/6", "4/6", "5/6", "6/6"]
+
+
+def test_sample_regex_mask_hex_and_escaping():
+    import random
+    rng = random.Random(42)
+
+    # 1. Hexadecimal token '%' (0-9, A-F)
+    hex_sample = sample_regex_mask("0x%%%%", rng)
+    assert hex_sample.startswith("0x")
+    assert len(hex_sample) == 6
+    for c in hex_sample[2:]:
+        assert c in "0123456789ABCDEF"
+
+    # 2. Escaped special tokens '\#', '\?', '\@', '\*', '\%', '\H'
+    escaped_sample = sample_regex_mask(r"\#\?\@\*\%\Hello", rng)
+    assert escaped_sample == "#?@*%Hello"
+
+    # 3. Escaped backslash '\\'
+    path_sample = sample_regex_mask(r"C:\\DIR\\\#-####", rng)
+    assert path_sample.startswith(r"C:\DIR\#-")
+    assert len(path_sample) == 13
+    assert path_sample[-4:].isdigit()
+
+    # 4. Trailing backslash
+    trailing_sample = sample_regex_mask("TEST-\\\\", rng)
+    assert trailing_sample == "TEST-\\"
+
+    single_trailing = sample_regex_mask("TEST-\\", rng)
+    assert single_trailing == "TEST-\\"
+
+    # 5. Table generator end-to-end integration test with masked string
+    table = {
+        "name": "hex_table",
+        "archetype": "record",
+        "status": "valid",
+        "fields": [
+            {
+                "name": "mac_addr",
+                "role": "randomized",
+                "type": "string",
+                "config": {"mask": r"MAC-\#%%-\#%%-\#%%"},
+                "active": True,
+            }
+        ],
+    }
+    gen = TableGenerator(table, [], seed=77, chunk_size=5)
+    df = next(gen.generate_record_chunks(5))
+    macs = df["mac_addr"].to_list()
+    assert len(macs) == 5
+    for m in macs:
+        assert m.startswith("MAC-#")
+        parts = m.split("-")
+        assert len(parts) == 4
+        assert parts[0] == "MAC"
+        for p in parts[1:]:
+            assert p.startswith("#")
+            assert len(p) == 3
+            assert all(c in "0123456789ABCDEF" for c in p[1:])
+

frontend/src/api/index.ts

@@ -81,6 +81,7 @@ export async function createTable(
     archetype: string;
     format?: string;
     row_limit?: number;
+    exclude_if?: string | null;
     fields?: any[];
     columns?: string[];
     data?: any[][];

frontend/src/components/CollectionEditor.tsx

@@ -5,6 +5,10 @@ import {
   Trash,
   CheckCircle,
   DownloadSimple,
+  ArrowsLeftRight,
+  Clipboard,
+  Check,
+  X,
   Lock,
   Scales,
 } from '@phosphor-icons/react';
@@ -23,6 +27,65 @@ interface CollectionItem {
   weight: number;
 }
 
+function parseCollectionTXT(text: string): CollectionItem[] {
+  const lines = text.split(/\r?\n/);
+  const items: CollectionItem[] = [];
+  for (const line of lines) {
+    const trimmed = line.trim();
+    if (trimmed) {
+      items.push({ value: trimmed, weight: 1.0 });
+    }
+  }
+  return items;
+}
+
+function parseCollectionJSON(jsonStr: string): CollectionItem[] {
+  const parsed = JSON.parse(jsonStr);
+  let rawList: any[] = [];
+  if (Array.isArray(parsed)) {
+    rawList = parsed;
+  } else if (parsed && Array.isArray(parsed.data)) {
+    rawList = parsed.data;
+  } else {
+    throw new Error("Invalid collection JSON format. Expected array or '{ data: [...] }'.");
+  }
+
+  const items: CollectionItem[] = [];
+  for (const row of rawList) {
+    if (Array.isArray(row)) {
+      const val = row[0] !== undefined ? String(row[0]).trim() : '';
+      const w = row[1] !== undefined ? parseFloat(row[1]) || 1.0 : 1.0;
+      if (val) items.push({ value: val, weight: Math.max(0.01, w) });
+    } else if (typeof row === 'object' && row !== null) {
+      const val = String((row as any).value ?? '').trim();
+      const w = parseFloat((row as any).weight) || 1.0;
+      if (val) items.push({ value: val, weight: Math.max(0.01, w) });
+    } else {
+      const val = String(row).trim();
+      if (val) items.push({ value: val, weight: 1.0 });
+    }
+  }
+  return items;
+}
+
+function parseImportContent(text: string): { items: CollectionItem[]; isUniform: boolean } {
+  const trimmed = text.trim();
+  if (trimmed.startsWith('{') || trimmed.startsWith('[')) {
+    const parsedItems = parseCollectionJSON(trimmed);
+    if (parsedItems.length === 0) {
+      throw new Error('No valid items found in JSON.');
+    }
+    const allUniform = parsedItems.every((it) => it.weight === 1.0);
+    return { items: parsedItems, isUniform: allUniform };
+  } else {
+    const parsedItems = parseCollectionTXT(trimmed);
+    if (parsedItems.length === 0) {
+      throw new Error('No items found in plain text input.');
+    }
+    return { items: parsedItems, isUniform: true };
+  }
+}
+
 export const CollectionEditor: React.FC<CollectionEditorProps> = ({
   table,
   onBack,
@@ -64,8 +127,78 @@ export const CollectionEditor: React.FC<CollectionEditorProps> = ({
   const [validationError, setValidationError] = useState<string | null>(null);
   const [successMessage, setSuccessMessage] = useState<string | null>(null);
 
+  // Import / Export Modal state
+  const [isIoModalOpen, setIsIoModalOpen] = useState(false);
+  const [ioTab, setIoTab] = useState<'export' | 'import'>('export');
+  const [importText, setImportText] = useState('');
+  const [importError, setImportError] = useState<string | null>(null);
+  const [copied, setCopied] = useState(false);
+  const [showConfirmGuard, setShowConfirmGuard] = useState(false);
+
   const isLocked = table.status === 'valid' && (table.referencing_tables?.length || 0) > 0;
 
+  const exportPayload = {
+    archetype: 'collection',
+    format: table.format || 'csv',
+    data: items.map((it) => [it.value, it.weight]),
+  };
+  const exportJsonString = JSON.stringify(exportPayload, null, 2);
+
+  const handleCopyExport = async () => {
+    try {
+      await navigator.clipboard.writeText(exportJsonString);
+      setCopied(true);
+      setTimeout(() => setCopied(false), 2000);
+    } catch (err) {
+      console.error('Failed to copy to clipboard', err);
+    }
+  };
+
+  const handleDownloadJson = () => {
+    const blob = new Blob([exportJsonString], { type: 'application/json' });
+    const url = URL.createObjectURL(blob);
+    const a = document.createElement('a');
+    a.href = url;
+    a.download = `${(table.slug || table.name).toLowerCase().replace(/\s+/g, '_')}_collection.json`;
+    a.click();
+    URL.revokeObjectURL(url);
+  };
+
+  const handleTriggerImport = () => {
+    if (isLocked) {
+      setImportError(
+        `Cannot alter collection: table is actively referenced by ${table.referencing_tables?.join(', ')}.`
+      );
+      return;
+    }
+    if (!importText.trim()) {
+      setImportError('Please enter items as JSON or line-separated text.');
+      return;
+    }
+    try {
+      parseImportContent(importText);
+      setImportError(null);
+      setShowConfirmGuard(true);
+    } catch (err: any) {
+      setImportError(err.message || 'Failed to parse import input.');
+    }
+  };
+
+  const handleConfirmImport = () => {
+    try {
+      const res = parseImportContent(importText);
+      setItems(res.items);
+      setIsUniformMode(res.isUniform);
+      setShowConfirmGuard(false);
+      setIsIoModalOpen(false);
+      setImportText('');
+      setSuccessMessage(`Imported ${res.items.length} items. Click 'Save Draft' to persist.`);
+    } catch (err: any) {
+      setImportError(err.message || 'Failed to import items.');
+      setShowConfirmGuard(false);
+    }
+  };
+
   const handleAddItem = () => {
     const newItem: CollectionItem = {
       value: `Item ${items.length + 1}`,
@@ -106,17 +239,20 @@ export const CollectionEditor: React.FC<CollectionEditorProps> = ({
   };
 
   const serializeData = (): any[][] => {
-    return items.map((it) => [it.value, it.weight]);
+    const cleanItems = items.map((it) => ({ ...it, value: it.value.trim() }));
+    setItems(cleanItems);
+    return cleanItems.map((it) => [it.value, it.weight]);
   };
 
   const handleSave = async () => {
     setSaving(true);
     setValidationError(null);
     setSuccessMessage(null);
+    const cleanData = serializeData();
     try {
       const updated = await updateTable(table.id, {
         columns: ['value', 'weight'],
-        data: serializeData(),
+        data: cleanData,
       });
       onTableUpdated(updated);
       setSuccessMessage('Collection draft saved.');
@@ -131,10 +267,11 @@ export const CollectionEditor: React.FC<CollectionEditorProps> = ({
     setValidating(true);
     setValidationError(null);
     setSuccessMessage(null);
+    const cleanData = serializeData();
     try {
       await updateTable(table.id, {
         columns: ['value', 'weight'],
-        data: serializeData(),
+        data: cleanData,
       });
       const result = await validateTable(table.id);
       if (result.success) {
@@ -196,6 +333,18 @@ export const CollectionEditor: React.FC<CollectionEditorProps> = ({
 
         {/* Action Buttons */}
         <div className="flex items-center space-x-3">
+          <button
+            onClick={() => {
+              setIsIoModalOpen(true);
+              setIoTab('export');
+              setImportError(null);
+            }}
+            className="flex items-center space-x-1.5 text-xs font-medium px-3.5 py-2 border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] hover:border-[#000000] dark:hover:border-[#FFFFFF] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+            title="Import or Export Collection"
+          >
+            <ArrowsLeftRight size={14} />
+            <span>Import / Export</span>
+          </button>
           <button
             onClick={() => onGenerate('csv')}
             disabled={table.status !== 'valid'}
@@ -302,7 +451,7 @@ export const CollectionEditor: React.FC<CollectionEditorProps> = ({
               {items.length === 0 ? (
                 <tr>
                   <td colSpan={4} className="py-12 text-center text-[#4F6170] dark:text-[#ADBFCE] text-xs font-mono">
-                    No items in collection. Click "Add Item" to add values.
+                    No items in collection. Click "Add Item" or "Import" to add values.
                   </td>
                 </tr>
               ) : (
@@ -364,6 +513,167 @@ export const CollectionEditor: React.FC<CollectionEditorProps> = ({
           </table>
         </div>
       </div>
+
+      {/* Import / Export Modal */}
+      {isIoModalOpen && (
+        <div className="fixed inset-0 z-50 flex items-center justify-center bg-black/40 backdrop-blur-xs p-4">
+          <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] w-full max-w-2xl p-6 space-y-4 shadow-xl">
+            {/* Header: Horizontal tabs on top-left, Close button on right */}
+            <div className="flex items-center justify-between border-b border-[#DEE3E8] dark:border-[#4F6170] pb-3">
+              <div className="flex items-center space-x-2">
+                <button
+                  type="button"
+                  onClick={() => {
+                    setIoTab('export');
+                    setImportError(null);
+                  }}
+                  className={`px-3 py-1.5 text-xs font-mono rounded-[6px] border transition-colors ${
+                    ioTab === 'export'
+                      ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000]'
+                      : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#0E1317]'
+                  }`}
+                >
+                  Export
+                </button>
+                <button
+                  type="button"
+                  onClick={() => {
+                    setIoTab('import');
+                    setImportError(null);
+                  }}
+                  className={`px-3 py-1.5 text-xs font-mono rounded-[6px] border transition-colors ${
+                    ioTab === 'import'
+                      ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000]'
+                      : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#0E1317]'
+                  }`}
+                >
+                  Import
+                </button>
+              </div>
+
+              <button
+                type="button"
+                onClick={() => {
+                  setIsIoModalOpen(false);
+                  setShowConfirmGuard(false);
+                  setImportError(null);
+                }}
+                className="p-1.5 rounded-[6px] border border-[#DEE3E8] dark:border-[#4F6170] text-[#4F6170] dark:text-[#ADBFCE] hover:text-[#000000] dark:hover:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] transition-colors"
+              >
+                <X size={15} />
+              </button>
+            </div>
+
+            {/* Tab 1: Export */}
+            {ioTab === 'export' && (
+              <div className="space-y-4">
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono">
+                  Collection schema definition (name excluded). Suitable for version control and migration.
+                </p>
+                <textarea
+                  readOnly
+                  rows={12}
+                  value={exportJsonString}
+                  className="w-full px-3 py-2 text-xs font-mono border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] bg-[#F4FAFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none select-all"
+                />
+                <div className="flex items-center justify-end space-x-3 pt-2">
+                  <button
+                    type="button"
+                    onClick={handleCopyExport}
+                    className="flex items-center space-x-1.5 text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+                  >
+                    {copied ? <Check size={14} weight="bold" /> : <Clipboard size={14} />}
+                    <span>{copied ? 'Copied!' : 'Copy to Clipboard'}</span>
+                  </button>
+                  <button
+                    type="button"
+                    onClick={handleDownloadJson}
+                    className="flex items-center space-x-1.5 text-xs font-medium px-4 py-2 bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000] rounded-[6px] hover:bg-[#525456] dark:hover:bg-[#DEE3E8] transition-all"
+                  >
+                    <DownloadSimple size={14} weight="bold" />
+                    <span>Download .json</span>
+                  </button>
+                </div>
+              </div>
+            )}
+
+            {/* Tab 2: Import */}
+            {ioTab === 'import' && (
+              <div className="space-y-4">
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono">
+                  Paste structured JSON or plain text (single line = single item).
+                </p>
+                <textarea
+                  rows={12}
+                  value={importText}
+                  disabled={isLocked}
+                  onChange={(e) => setImportText(e.target.value)}
+                  placeholder={`Option A (Plain text):\nItem One\nItem Two\nItem Three\n\nOption B (JSON):\n{ "archetype": "collection", "data": [["Item One", 1.0], ["Item Two", 2.5]] }`}
+                  className="w-full px-3 py-2 text-xs font-mono border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none focus:border-[#000000] dark:focus:border-[#ADBFCE] disabled:opacity-50"
+                />
+
+                {importError && (
+                  <div className="p-3 bg-[#FDEBEC] dark:bg-[#321214] border border-[#DEE3E8] dark:border-[#9F2F2D] rounded-[6px] text-xs font-mono text-[#9F2F2D] dark:text-[#FDEBEC]">
+                    {importError}
+                  </div>
+                )}
+
+                <div className="flex items-center justify-end space-x-3 pt-2">
+                  <button
+                    type="button"
+                    onClick={() => {
+                      setIsIoModalOpen(false);
+                      setImportText('');
+                      setImportError(null);
+                    }}
+                    className="text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF]"
+                  >
+                    Cancel
+                  </button>
+                  <button
+                    type="button"
+                    disabled={isLocked || !importText.trim()}
+                    onClick={handleTriggerImport}
+                    className="text-xs font-medium px-4 py-2 bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000] rounded-[6px] hover:bg-[#525456] dark:hover:bg-[#DEE3E8] disabled:opacity-40 transition-all"
+                  >
+                    Import
+                  </button>
+                </div>
+              </div>
+            )}
+          </div>
+
+          {/* Confirmation Guard Dialog */}
+          {showConfirmGuard && (
+            <div className="fixed inset-0 z-60 flex items-center justify-center bg-[#000000]/60 backdrop-blur-xs p-4">
+              <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] max-w-md w-full p-6 shadow-2xl space-y-4">
+                <h3 className="text-base font-semibold text-[#000000] dark:text-[#FFFFFF]">
+                  Confirm Collection Overwrite
+                </h3>
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono leading-relaxed">
+                  Are you sure you want to import? This will overwrite the current columns/data in this table.
+                </p>
+                <div className="flex items-center justify-end space-x-3 pt-3 border-t border-[#DEE3E8] dark:border-[#4F6170]">
+                  <button
+                    type="button"
+                    onClick={() => setShowConfirmGuard(false)}
+                    className="px-3.5 py-2 text-xs font-medium border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] transition-colors"
+                  >
+                    Cancel
+                  </button>
+                  <button
+                    type="button"
+                    onClick={handleConfirmImport}
+                    className="px-4 py-2 text-xs font-medium bg-[#9F2F2D] text-[#FFFFFF] rounded-[6px] hover:bg-[#7D2523] transition-colors"
+                  >
+                    Overwrite & Import
+                  </button>
+                </div>
+              </div>
+            </div>
+          )}
+        </div>
+      )}
     </div>
   );
 };

frontend/src/components/ProjectDetail.tsx

@@ -8,8 +8,11 @@ import {
   Lock,
   Sliders,
   FileCode,
+  Copy,
+  PencilSimple,
+  X,
 } from '@phosphor-icons/react';
-import { createTable, deleteTable, invalidateTable, validateTable } from '../api';
+import { createTable, deleteTable, invalidateTable, updateTable, validateTable } from '../api';
 import { sanitizeSlugInput, slugify } from '../hooks/useRoute';
 import type { Table, TableArchetype } from '../types';
 
@@ -46,6 +49,13 @@ export const ProjectDetail: React.FC<ProjectDetailProps> = ({
   const [isCreating, setIsCreating] = useState(false);
   const [actionError, setActionError] = useState<string | null>(null);
 
+  // Rename Table Modal state
+  const [renamingTable, setRenamingTable] = useState<Table | null>(null);
+  const [renameName, setRenameName] = useState('');
+  const [renameSlug, setRenameSlug] = useState('');
+  const [renameError, setRenameError] = useState<string | null>(null);
+  const [isRenaming, setIsRenaming] = useState(false);
+
   const handleNameChange = (val: string) => {
     setNewTableName(val);
     if (!newTableSlug || newTableSlug === slugify(newTableName)) {
@@ -53,6 +63,85 @@ export const ProjectDetail: React.FC<ProjectDetailProps> = ({
     }
   };
 
+  const handleRenameNameChange = (val: string) => {
+    setRenameName(val);
+    if (!renameSlug || renameSlug === slugify(renameName)) {
+      setRenameSlug(slugify(val));
+    }
+  };
+
+  const handleDuplicateTable = async (table: Table) => {
+    setActionError(null);
+    const existingNames = new Set(project.tables.map((t) => t.name.toLowerCase()));
+    const existingSlugs = new Set(project.tables.map((t) => (t.slug || t.name).toLowerCase()));
+
+    const baseName = `${table.name}_copy`;
+    let newName = baseName;
+    let counter = 2;
+    while (existingNames.has(newName.toLowerCase())) {
+      newName = `${table.name}_copy_${counter}`;
+      counter++;
+    }
+
+    const baseSlug = `${table.slug || slugify(table.name)}-copy`;
+    let newSlug = sanitizeSlugInput(baseSlug);
+    counter = 2;
+    while (existingSlugs.has(newSlug.toLowerCase())) {
+      newSlug = sanitizeSlugInput(`${table.slug || slugify(table.name)}-copy-${counter}`);
+      counter++;
+    }
+
+    try {
+      await createTable(project.id, {
+        name: newName,
+        slug: newSlug,
+        archetype: table.archetype,
+        format: table.format,
+        row_limit: table.row_limit ?? undefined,
+        exclude_if: table.exclude_if ?? undefined,
+        fields: table.fields ? JSON.parse(JSON.stringify(table.fields)) : [],
+        columns: table.columns ? [...table.columns] : [],
+        data: table.data ? JSON.parse(JSON.stringify(table.data)) : [],
+      });
+      onRefreshProject();
+    } catch (err: any) {
+      setActionError(err.message || 'Failed to duplicate table');
+    }
+  };
+
+  const handleRenameSubmit = async (e: React.FormEvent) => {
+    e.preventDefault();
+    if (!renamingTable) return;
+    const cleanName = renameName.trim();
+    const cleanSlug = renameSlug.trim();
+    if (!cleanName) {
+      setRenameError('Table name cannot be empty');
+      return;
+    }
+    if (!cleanSlug) {
+      setRenameError('Table slug cannot be empty');
+      return;
+    }
+    if (!/^[a-z0-9-]+$/.test(cleanSlug)) {
+      setRenameError('Slug can only contain lowercase alphanumerics and hyphens (a-z, 0-9, -).');
+      return;
+    }
+    setIsRenaming(true);
+    setRenameError(null);
+    try {
+      await updateTable(renamingTable.id, {
+        name: cleanName,
+        slug: cleanSlug,
+      });
+      setRenamingTable(null);
+      onRefreshProject();
+    } catch (err: any) {
+      setRenameError(err.message || 'Failed to rename table');
+    } finally {
+      setIsRenaming(false);
+    }
+  };
+
   const handleCreateTable = async (e: React.FormEvent) => {
     e.preventDefault();
     if (!newTableName.trim()) return;
@@ -219,16 +308,41 @@ export const ProjectDetail: React.FC<ProjectDetailProps> = ({
               className="bg-[#FFFFFF] dark:bg-[#0E1317] border border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] p-5 flex flex-col justify-between hover:border-[#000000] dark:hover:border-[#ADBFCE] transition-all"
             >
               <div className="space-y-3">
-                {/* Table Header with Clickable Title */}
+                {/* Table Header with Clickable Title & Rename Action */}
                 <div className="flex items-start justify-between">
                   <div className="space-y-1">
-                    <h3
+                    <div className="flex items-center space-x-1.5">
-                      onClick={() => onSelectTable(table)}
+                      <h3
-                      className="font-semibold text-base text-[#000000] dark:text-[#FFFFFF] hover:text-[#4F6170] dark:hover:text-[#ADBFCE] hover:underline cursor-pointer transition-colors"
+                        onClick={() => onSelectTable(table)}
-                      title={`Open ${table.archetype === 'reference' ? 'Data Grid' : table.archetype === 'collection' ? 'Collection List' : 'DAG Schema'}`}
+                        className="font-semibold text-base text-[#000000] dark:text-[#FFFFFF] hover:text-[#4F6170] dark:hover:text-[#ADBFCE] hover:underline cursor-pointer transition-colors"
-                    >
+                        title={`Open ${table.archetype === 'reference' ? 'Data Grid' : table.archetype === 'collection' ? 'Collection List' : 'DAG Schema'}`}
-                      {table.name}
+                      >
-                    </h3>
+                        {table.name}
+                      </h3>
+                      <button
+                        type="button"
+                        disabled={isLocked}
+                        onClick={(e) => {
+                          e.stopPropagation();
+                          setRenamingTable(table);
+                          setRenameName(table.name);
+                          setRenameSlug(table.slug || slugify(table.name));
+                          setRenameError(null);
+                        }}
+                        className={`p-1 rounded-[4px] border border-transparent transition-colors ${
+                          isLocked
+                            ? 'opacity-30 cursor-not-allowed text-[#ADBFCE]'
+                            : 'text-[#4F6170] dark:text-[#ADBFCE] hover:border-[#DEE3E8] dark:hover:border-[#4F6170] hover:text-[#000000] dark:hover:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22]'
+                        }`}
+                        title={
+                          isLocked
+                            ? `Cannot rename table: actively referenced by ${table.referencing_tables.join(', ')}`
+                            : 'Rename table'
+                        }
+                      >
+                        <PencilSimple size={13} />
+                      </button>
+                    </div>
                     <div className="flex items-center space-x-2">
                       <span
                         className={`text-[11px] uppercase font-mono px-2 py-0.5 rounded-full border border-[#DEE3E8] dark:border-[#4F6170] ${
@@ -324,6 +438,13 @@ export const ProjectDetail: React.FC<ProjectDetailProps> = ({
                 </div>
 
                 <div className="flex items-center space-x-2">
+                  <button
+                    onClick={() => handleDuplicateTable(table)}
+                    className="p-1.5 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+                    title="Duplicate table"
+                  >
+                    <Copy size={14} />
+                  </button>
                   <button
                     onClick={() => onGenerateTable(table)}
                     disabled={table.status !== 'valid'}
@@ -461,6 +582,78 @@ export const ProjectDetail: React.FC<ProjectDetailProps> = ({
           </div>
         </div>
       )}
+      {/* Rename Table Modal */}
+      {renamingTable && (
+        <div className="fixed inset-0 z-50 flex items-center justify-center bg-black/40 backdrop-blur-xs p-4">
+          <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] w-full max-w-md p-6 space-y-5 shadow-xl">
+            <div className="flex items-center justify-between border-b border-[#DEE3E8] dark:border-[#4F6170] pb-3">
+              <h3 className="text-base font-semibold text-[#000000] dark:text-[#FFFFFF]">Rename Table</h3>
+              <button
+                type="button"
+                onClick={() => {
+                  setRenamingTable(null);
+                  setRenameError(null);
+                }}
+                className="p-1.5 rounded-[6px] border border-[#DEE3E8] dark:border-[#4F6170] text-[#4F6170] dark:text-[#ADBFCE] hover:text-[#000000] dark:hover:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22]"
+              >
+                <X size={14} />
+              </button>
+            </div>
+
+            <form onSubmit={handleRenameSubmit} className="space-y-4">
+              <div>
+                <label className="block text-xs font-medium text-[#4F6170] dark:text-[#ADBFCE] mb-1">Table Name</label>
+                <input
+                  type="text"
+                  required
+                  value={renameName}
+                  onChange={(e) => handleRenameNameChange(e.target.value)}
+                  className="w-full px-3 py-2 text-xs font-mono border border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none focus:border-[#000000] dark:focus:border-[#ADBFCE]"
+                />
+              </div>
+
+              <div>
+                <label className="block text-xs font-medium text-[#4F6170] dark:text-[#ADBFCE] mb-1">
+                  Table Slug (Alphanumerics & hyphens only)
+                </label>
+                <input
+                  type="text"
+                  required
+                  value={renameSlug}
+                  onChange={(e) => setRenameSlug(sanitizeSlugInput(e.target.value))}
+                  className="w-full px-3 py-2 text-xs font-mono border border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none focus:border-[#000000] dark:focus:border-[#ADBFCE]"
+                />
+              </div>
+
+              {renameError && (
+                <div className="p-3 bg-[#FDEBEC] dark:bg-[#321214] border border-[#DEE3E8] dark:border-[#9F2F2D] rounded-[6px] text-xs font-mono text-[#9F2F2D] dark:text-[#FDEBEC]">
+                  {renameError}
+                </div>
+              )}
+
+              <div className="flex items-center justify-end space-x-3 pt-3 border-t border-[#DEE3E8] dark:border-[#4F6170]">
+                <button
+                  type="button"
+                  onClick={() => {
+                    setRenamingTable(null);
+                    setRenameError(null);
+                  }}
+                  className="text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF]"
+                >
+                  Cancel
+                </button>
+                <button
+                  type="submit"
+                  disabled={isRenaming}
+                  className="text-xs font-medium px-4 py-2 bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000] rounded-[6px] hover:bg-[#525456] dark:hover:bg-[#DEE3E8] disabled:opacity-50"
+                >
+                  {isRenaming ? 'Saving...' : 'Save Changes'}
+                </button>
+              </div>
+            </form>
+          </div>
+        </div>
+      )}
     </div>
   );
 };

frontend/src/components/ReferenceSpreadsheet.tsx

@@ -1,5 +1,17 @@
-import React, { useState } from 'react';
+import React, { useRef, useState } from 'react';
-import { Plus, Trash, CheckCircle, DownloadSimple, ArrowLeft, Lock } from '@phosphor-icons/react';
+import {
+  Plus,
+  Trash,
+  CheckCircle,
+  DownloadSimple,
+  ArrowsLeftRight,
+  Clipboard,
+  Check,
+  X,
+  ArrowLeft,
+  Lock,
+  Warning,
+} from '@phosphor-icons/react';
 import { updateTable, validateTable } from '../api';
 import type { Table } from '../types';
 
@@ -10,6 +22,78 @@ interface ReferenceSpreadsheetProps {
   onGenerate: (format: 'csv' | 'parquet') => void;
 }
 
+function parseCSV(text: string): { columns: string[]; rows: string[][] } {
+  const lines: string[][] = [];
+  let row: string[] = [];
+  let current = '';
+  let inQuotes = false;
+
+  for (let i = 0; i < text.length; i++) {
+    const char = text[i];
+    const nextChar = text[i + 1];
+
+    if (char === '"') {
+      if (inQuotes && nextChar === '"') {
+        current += '"';
+        i++;
+      } else {
+        inQuotes = !inQuotes;
+      }
+    } else if (char === ',' && !inQuotes) {
+      row.push(current);
+      current = '';
+    } else if ((char === '\r' || char === '\n') && !inQuotes) {
+      if (char === '\r' && nextChar === '\n') {
+        i++;
+      }
+      row.push(current);
+      current = '';
+      if (row.length > 0 && (row.length > 1 || row[0] !== '')) {
+        lines.push(row);
+      }
+      row = [];
+    } else {
+      current += char;
+    }
+  }
+
+  if (current !== '' || row.length > 0) {
+    row.push(current);
+    if (row.length > 0 && (row.length > 1 || row[0] !== '')) {
+      lines.push(row);
+    }
+  }
+
+  if (lines.length === 0) {
+    return { columns: [], rows: [] };
+  }
+
+  const columns = lines[0].map((c) => c.trim());
+  const rows = lines.slice(1).map((r) => {
+    const padded = [...r];
+    while (padded.length < columns.length) padded.push('');
+    return padded.slice(0, columns.length);
+  });
+
+  return { columns, rows };
+}
+
+function parseReferenceJSON(jsonStr: string): { columns: string[]; rows: any[][] } {
+  const parsed = JSON.parse(jsonStr);
+  if (parsed && Array.isArray(parsed.columns) && Array.isArray(parsed.data)) {
+    return {
+      columns: parsed.columns.map((c: any) => String(c).trim()),
+      rows: parsed.data,
+    };
+  }
+  throw new Error("Invalid reference JSON format. Expected '{ columns: [...], data: [[...]] }'.");
+}
+
+function areColumnsEqual(colsA: string[], colsB: string[]): boolean {
+  if (colsA.length !== colsB.length) return false;
+  return colsA.every((c, i) => c === colsB[i]);
+}
+
 export const ReferenceSpreadsheet: React.FC<ReferenceSpreadsheetProps> = ({
   table,
   onBack,
@@ -23,8 +107,168 @@ export const ReferenceSpreadsheet: React.FC<ReferenceSpreadsheetProps> = ({
   const [validationError, setValidationError] = useState<string | null>(null);
   const [successMessage, setSuccessMessage] = useState<string | null>(null);
 
+  // Column mismatch modal state
+  const [pendingImport, setPendingImport] = useState<{
+    columns: string[];
+    rows: any[][];
+    fileName: string;
+  } | null>(null);
+
+  // Import / Export Modal state
+  const [isIoModalOpen, setIsIoModalOpen] = useState(false);
+  const [ioTab, setIoTab] = useState<'import-json' | 'csv' | 'export'>('export');
+  const [jsonImportText, setJsonImportText] = useState('');
+  const [jsonImportError, setJsonImportError] = useState<string | null>(null);
+  const [csvImportError, setCsvImportError] = useState<string | null>(null);
+  const [copied, setCopied] = useState(false);
+  const [pendingGuardImport, setPendingGuardImport] = useState<{
+    columns: string[];
+    rows: any[][];
+    sourceName: string;
+  } | null>(null);
+
+  const csvFileInputRef = useRef<HTMLInputElement>(null);
   const isLocked = table.status === 'valid' && table.referencing_tables.length > 0;
 
+  const exportPayload = {
+    archetype: 'reference',
+    format: table.format || 'csv',
+    columns,
+    data,
+  };
+  const exportJsonString = JSON.stringify(exportPayload, null, 2);
+
+  const handleCopyExport = async () => {
+    try {
+      await navigator.clipboard.writeText(exportJsonString);
+      setCopied(true);
+      setTimeout(() => setCopied(false), 2000);
+    } catch (err) {
+      console.error('Failed to copy', err);
+    }
+  };
+
+  const handleDownloadJson = () => {
+    const blob = new Blob([exportJsonString], { type: 'application/json' });
+    const url = URL.createObjectURL(blob);
+    const a = document.createElement('a');
+    a.href = url;
+    a.download = `${(table.slug || table.name).toLowerCase().replace(/\s+/g, '_')}_reference.json`;
+    a.click();
+    URL.revokeObjectURL(url);
+  };
+
+  const handleTriggerJsonImport = () => {
+    if (!jsonImportText.trim()) {
+      setJsonImportError('Please enter reference table JSON.');
+      return;
+    }
+    setJsonImportError(null);
+    try {
+      const res = parseReferenceJSON(jsonImportText);
+      const matches = areColumnsEqual(columns, res.columns);
+      if (matches) {
+        setPendingGuardImport({
+          columns: res.columns,
+          rows: res.rows,
+          sourceName: 'JSON specification',
+        });
+      } else {
+        if (isLocked) {
+          setJsonImportError(
+            `Cannot change table columns to [${res.columns.join(', ')}]: table is actively referenced by ${table.referencing_tables.join(', ')}.`
+          );
+          return;
+        }
+        setPendingImport({
+          columns: res.columns,
+          rows: res.rows,
+          fileName: 'JSON specification',
+        });
+      }
+    } catch (err: any) {
+      setJsonImportError(err.message || 'Invalid JSON format');
+    }
+  };
+
+  const handleCsvFileSelect = (e: React.ChangeEvent<HTMLInputElement>) => {
+    const file = e.target.files?.[0];
+    if (!file) return;
+    setCsvImportError(null);
+    const reader = new FileReader();
+    reader.onload = (event) => {
+      const content = event.target?.result as string;
+      if (!content || !content.trim()) {
+        setCsvImportError('Selected file is empty.');
+        return;
+      }
+      try {
+        const res = parseCSV(content);
+        if (res.columns.length === 0) {
+          setCsvImportError('No columns found in the CSV file.');
+          return;
+        }
+        const matches = areColumnsEqual(columns, res.columns);
+        if (matches) {
+          setPendingGuardImport({
+            columns: res.columns,
+            rows: res.rows,
+            sourceName: file.name,
+          });
+        } else {
+          if (isLocked) {
+            setCsvImportError(
+              `Cannot change table columns to [${res.columns.join(', ')}]: table is actively referenced by ${table.referencing_tables.join(', ')}.`
+            );
+            return;
+          }
+          setPendingImport({
+            columns: res.columns,
+            rows: res.rows,
+            fileName: file.name,
+          });
+        }
+      } catch (err: any) {
+        setCsvImportError(err.message || 'Failed to parse CSV file.');
+      } finally {
+        if (csvFileInputRef.current) csvFileInputRef.current.value = '';
+      }
+    };
+    reader.onerror = () => {
+      setCsvImportError('Failed to read file.');
+      if (csvFileInputRef.current) csvFileInputRef.current.value = '';
+    };
+    reader.readAsText(file);
+  };
+
+  const handleConfirmGuardImport = () => {
+    if (!pendingGuardImport) return;
+    setColumns(pendingGuardImport.columns);
+    setData(pendingGuardImport.rows);
+    setSuccessMessage(
+      `Imported ${pendingGuardImport.rows.length} records from ${pendingGuardImport.sourceName}. Click 'Save Draft' to persist.`
+    );
+    setPendingGuardImport(null);
+    setIsIoModalOpen(false);
+    setJsonImportText('');
+  };
+
+  const confirmPendingImport = () => {
+    if (!pendingImport) return;
+    setColumns(pendingImport.columns);
+    setData(pendingImport.rows);
+    setSuccessMessage(
+      `Replaced columns and imported ${pendingImport.rows.length} records from ${pendingImport.fileName}. Click 'Save Draft' to persist.`
+    );
+    setPendingImport(null);
+    setIsIoModalOpen(false);
+    setJsonImportText('');
+  };
+
+  const cancelPendingImport = () => {
+    setPendingImport(null);
+  };
+
   const handleAddColumn = () => {
     const colName = prompt('Enter new column name:');
     if (!colName || !colName.trim()) return;
@@ -71,14 +315,25 @@ export const ReferenceSpreadsheet: React.FC<ReferenceSpreadsheetProps> = ({
     setData(updated);
   };
 
+  const sanitizeSpreadsheet = () => {
+    const cleanColumns = columns.map((c) => c.trim());
+    const cleanData = data.map((row) =>
+      row.map((cell) => (typeof cell === 'string' ? cell.trim() : cell))
+    );
+    setColumns(cleanColumns);
+    setData(cleanData);
+    return { cleanColumns, cleanData };
+  };
+
   const handleSave = async () => {
     setSaving(true);
     setValidationError(null);
     setSuccessMessage(null);
+    const { cleanColumns, cleanData } = sanitizeSpreadsheet();
     try {
       const updated = await updateTable(table.id, {
-        columns,
+        columns: cleanColumns,
-        data,
+        data: cleanData,
       });
       onTableUpdated(updated);
       setSuccessMessage('Reference table data saved.');
@@ -93,8 +348,9 @@ export const ReferenceSpreadsheet: React.FC<ReferenceSpreadsheetProps> = ({
     setValidating(true);
     setValidationError(null);
     setSuccessMessage(null);
+    const { cleanColumns, cleanData } = sanitizeSpreadsheet();
     try {
-      await updateTable(table.id, { columns, data });
+      await updateTable(table.id, { columns: cleanColumns, data: cleanData });
       const result = await validateTable(table.id);
       if (result.success) {
         setSuccessMessage('Table validated and locked for referencing.');
@@ -153,6 +409,19 @@ export const ReferenceSpreadsheet: React.FC<ReferenceSpreadsheetProps> = ({
 
         {/* Action Buttons */}
         <div className="flex items-center space-x-3">
+          <button
+            onClick={() => {
+              setIsIoModalOpen(true);
+              setIoTab('export');
+              setJsonImportError(null);
+              setCsvImportError(null);
+            }}
+            className="flex items-center space-x-1.5 text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] hover:border-[#000000] dark:hover:border-[#FFFFFF] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+            title="Import or Export Reference Table"
+          >
+            <ArrowsLeftRight size={14} />
+            <span>Import / Export</span>
+          </button>
           <button
             onClick={() => onGenerate('csv')}
             disabled={table.status !== 'valid'}
@@ -261,7 +530,7 @@ export const ReferenceSpreadsheet: React.FC<ReferenceSpreadsheetProps> = ({
                     colSpan={columns.length + 2}
                     className="py-12 text-center text-[#4F6170] dark:text-[#ADBFCE] text-xs font-mono"
                   >
-                    No rows in catalog. Click "Add Row" to populate reference entries.
+                    No rows in catalog. Click "Add Row" or "Import" to populate reference entries.
                   </td>
                 </tr>
               ) : (
@@ -298,6 +567,280 @@ export const ReferenceSpreadsheet: React.FC<ReferenceSpreadsheetProps> = ({
           </table>
         </div>
       </div>
+
+      {/* Import / Export Modal */}
+      {isIoModalOpen && (
+        <div className="fixed inset-0 z-50 flex items-center justify-center bg-black/40 backdrop-blur-xs p-4">
+          <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] w-full max-w-2xl p-6 space-y-4 shadow-xl">
+            {/* Header: 3 horizontal tabs on top-left, Close button on right */}
+            <div className="flex items-center justify-between border-b border-[#DEE3E8] dark:border-[#4F6170] pb-3">
+              <div className="flex items-center space-x-2">
+                <button
+                  type="button"
+                  onClick={() => {
+                    setIoTab('import-json');
+                    setJsonImportError(null);
+                  }}
+                  className={`px-3 py-1.5 text-xs font-mono rounded-[6px] border transition-colors ${
+                    ioTab === 'import-json'
+                      ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000]'
+                      : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#0E1317]'
+                  }`}
+                >
+                  Import JSON
+                </button>
+                <button
+                  type="button"
+                  onClick={() => {
+                    setIoTab('csv');
+                    setCsvImportError(null);
+                  }}
+                  className={`px-3 py-1.5 text-xs font-mono rounded-[6px] border transition-colors ${
+                    ioTab === 'csv'
+                      ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000]'
+                      : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#0E1317]'
+                  }`}
+                >
+                  CSV Input
+                </button>
+                <button
+                  type="button"
+                  onClick={() => {
+                    setIoTab('export');
+                  }}
+                  className={`px-3 py-1.5 text-xs font-mono rounded-[6px] border transition-colors ${
+                    ioTab === 'export'
+                      ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000]'
+                      : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#0E1317]'
+                  }`}
+                >
+                  Export JSON
+                </button>
+              </div>
+
+              <button
+                type="button"
+                onClick={() => {
+                  setIsIoModalOpen(false);
+                  setPendingGuardImport(null);
+                  setJsonImportError(null);
+                  setCsvImportError(null);
+                }}
+                className="p-1.5 rounded-[6px] border border-[#DEE3E8] dark:border-[#4F6170] text-[#4F6170] dark:text-[#ADBFCE] hover:text-[#000000] dark:hover:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] transition-colors"
+              >
+                <X size={15} />
+              </button>
+            </div>
+
+            {/* Tab 1: Import JSON */}
+            {ioTab === 'import-json' && (
+              <div className="space-y-4">
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono">
+                  Paste reference table JSON containing columns and data matrix.
+                </p>
+                <textarea
+                  rows={12}
+                  value={jsonImportText}
+                  disabled={isLocked}
+                  onChange={(e) => setJsonImportText(e.target.value)}
+                  placeholder='{ "archetype": "reference", "columns": ["id", "label"], "data": [["1", "Option A"], ["2", "Option B"]] }'
+                  className="w-full px-3 py-2 text-xs font-mono border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none focus:border-[#000000] dark:focus:border-[#ADBFCE] disabled:opacity-50"
+                />
+
+                {jsonImportError && (
+                  <div className="p-3 bg-[#FDEBEC] dark:bg-[#321214] border border-[#DEE3E8] dark:border-[#9F2F2D] rounded-[6px] text-xs font-mono text-[#9F2F2D] dark:text-[#FDEBEC]">
+                    {jsonImportError}
+                  </div>
+                )}
+
+                <div className="flex items-center justify-end space-x-3 pt-2">
+                  <button
+                    type="button"
+                    onClick={() => {
+                      setIsIoModalOpen(false);
+                      setJsonImportText('');
+                      setJsonImportError(null);
+                    }}
+                    className="text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF]"
+                  >
+                    Cancel
+                  </button>
+                  <button
+                    type="button"
+                    disabled={isLocked || !jsonImportText.trim()}
+                    onClick={handleTriggerJsonImport}
+                    className="text-xs font-medium px-4 py-2 bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000] rounded-[6px] hover:bg-[#525456] dark:hover:bg-[#DEE3E8] disabled:opacity-40 transition-all"
+                  >
+                    Import
+                  </button>
+                </div>
+              </div>
+            )}
+
+            {/* Tab 2: CSV Input */}
+            {ioTab === 'csv' && (
+              <div className="space-y-4">
+                <input
+                  type="file"
+                  ref={csvFileInputRef}
+                  onChange={handleCsvFileSelect}
+                  accept=".csv"
+                  className="hidden"
+                />
+                <div className="border-2 border-dashed border-[#DEE3E8] dark:border-[#4F6170] rounded-[8px] p-8 flex flex-col items-center justify-center space-y-3 text-center">
+                  <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono max-w-sm">
+                    Upload a strictly formatted .csv file. The first row must define column headers.
+                  </p>
+                  <button
+                    type="button"
+                    disabled={isLocked}
+                    onClick={() => csvFileInputRef.current?.click()}
+                    className="px-4 py-2 text-xs font-medium border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors disabled:opacity-40"
+                  >
+                    Select .CSV File
+                  </button>
+                </div>
+
+                {csvImportError && (
+                  <div className="p-3 bg-[#FDEBEC] dark:bg-[#321214] border border-[#DEE3E8] dark:border-[#9F2F2D] rounded-[6px] text-xs font-mono text-[#9F2F2D] dark:text-[#FDEBEC]">
+                    {csvImportError}
+                  </div>
+                )}
+
+                <div className="flex items-center justify-end space-x-3 pt-2">
+                  <button
+                    type="button"
+                    onClick={() => {
+                      setIsIoModalOpen(false);
+                      setCsvImportError(null);
+                    }}
+                    className="text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF]"
+                  >
+                    Close
+                  </button>
+                </div>
+              </div>
+            )}
+
+            {/* Tab 3: Export JSON */}
+            {ioTab === 'export' && (
+              <div className="space-y-4">
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono">
+                  Reference catalog schema and records (name excluded). Suitable for version control and migration.
+                </p>
+                <textarea
+                  readOnly
+                  rows={12}
+                  value={exportJsonString}
+                  className="w-full px-3 py-2 text-xs font-mono border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] bg-[#F4FAFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none select-all"
+                />
+                <div className="flex items-center justify-end space-x-3 pt-2">
+                  <button
+                    type="button"
+                    onClick={handleCopyExport}
+                    className="flex items-center space-x-1.5 text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+                  >
+                    {copied ? <Check size={14} weight="bold" /> : <Clipboard size={14} />}
+                    <span>{copied ? 'Copied!' : 'Copy to Clipboard'}</span>
+                  </button>
+                  <button
+                    type="button"
+                    onClick={handleDownloadJson}
+                    className="flex items-center space-x-1.5 text-xs font-medium px-4 py-2 bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000] rounded-[6px] hover:bg-[#525456] dark:hover:bg-[#DEE3E8] transition-all"
+                  >
+                    <DownloadSimple size={14} weight="bold" />
+                    <span>Download .json</span>
+                  </button>
+                </div>
+              </div>
+            )}
+          </div>
+        </div>
+      )}
+
+      {/* Confirmation Guard Dialog (Matching Columns) */}
+      {pendingGuardImport && (
+        <div className="fixed inset-0 z-60 flex items-center justify-center bg-[#000000]/60 backdrop-blur-xs p-4">
+          <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] max-w-md w-full p-6 shadow-2xl space-y-4">
+            <h3 className="text-base font-semibold text-[#000000] dark:text-[#FFFFFF]">
+              Confirm Data Overwrite
+            </h3>
+            <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono leading-relaxed">
+              Are you sure you want to import? This will overwrite the current columns/data in this table.
+            </p>
+            <div className="flex items-center justify-end space-x-3 pt-3 border-t border-[#DEE3E8] dark:border-[#4F6170]">
+              <button
+                type="button"
+                onClick={() => setPendingGuardImport(null)}
+                className="px-3.5 py-2 text-xs font-medium border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] transition-colors"
+              >
+                Cancel
+              </button>
+              <button
+                type="button"
+                onClick={handleConfirmGuardImport}
+                className="px-4 py-2 text-xs font-medium bg-[#9F2F2D] text-[#FFFFFF] rounded-[6px] hover:bg-[#7D2523] transition-colors"
+              >
+                Overwrite & Import
+              </button>
+            </div>
+          </div>
+        </div>
+      )}
+
+      {/* Column Mismatch Warning Modal */}
+      {pendingImport && (
+        <div className="fixed inset-0 z-50 flex items-center justify-center bg-[#000000]/60 backdrop-blur-xs p-4">
+          <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] max-w-lg w-full p-6 shadow-xl space-y-4">
+            <div className="flex items-center space-x-3 text-[#956400] dark:text-[#ffd666]">
+              <Warning size={24} weight="bold" />
+              <h3 className="text-base font-semibold text-[#000000] dark:text-[#FFFFFF]">
+                Column Mismatch Warning
+              </h3>
+            </div>
+
+            <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] leading-relaxed font-mono">
+              The columns in the imported file do not match the current reference table columns (comparison is case-sensitive). Importing will replace your current column structure and all rows.
+            </p>
+
+            <div className="space-y-2 bg-[#F4FAFF] dark:bg-[#161C22] p-3 rounded-[6px] border border-[#DEE3E8] dark:border-[#4F6170] text-xs font-mono">
+              <div>
+                <span className="text-[#4F6170] dark:text-[#ADBFCE]">Current Columns: </span>
+                <span className="text-[#000000] dark:text-[#FFFFFF] font-bold">
+                  [{columns.join(', ')}]
+                </span>
+              </div>
+              <div>
+                <span className="text-[#4F6170] dark:text-[#ADBFCE]">Imported Columns: </span>
+                <span className="text-[#346538] dark:text-[#88d48e] font-bold">
+                  [{pendingImport.columns.join(', ')}]
+                </span>
+              </div>
+              <div className="pt-1 text-[#4F6170] dark:text-[#ADBFCE]">
+                Total Rows: <span className="font-bold text-[#000000] dark:text-[#FFFFFF]">{pendingImport.rows.length}</span>
+              </div>
+            </div>
+
+            <div className="flex items-center justify-end space-x-3 pt-2">
+              <button
+                type="button"
+                onClick={cancelPendingImport}
+                className="px-4 py-2 text-xs font-medium border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] transition-colors"
+              >
+                Cancel
+              </button>
+              <button
+                type="button"
+                onClick={confirmPendingImport}
+                className="px-4 py-2 text-xs font-medium bg-[#9F2F2D] text-[#FFFFFF] rounded-[6px] hover:bg-[#7D2523] transition-colors"
+              >
+                Proceed & Replace Columns
+              </button>
+            </div>
+          </div>
+        </div>
+      )}
     </div>
   );
 };
+

frontend/src/components/TableSchemaEditor.tsx

@@ -1,4 +1,4 @@
-import React, { useState } from 'react';
+import React, { useEffect, useRef, useState } from 'react';
 import {
   ArrowLeft,
   ArrowUp,
@@ -10,6 +10,12 @@ import {
   EyeSlash,
   Lock,
   DownloadSimple,
+  ArrowsLeftRight,
+  Clipboard,
+  Check,
+  X,
+  BracketsCurly,
+  MagnifyingGlass,
 } from '@phosphor-icons/react';
 import { updateTable, validateTable, previewTable } from '../api';
 import type { ColumnRole, ColumnType, Field, Table } from '../types';
@@ -23,6 +29,16 @@ interface TableSchemaEditorProps {
   onGenerate: (format: 'csv' | 'parquet') => void;
 }
 
+const BUILTIN_METADATA_VARIABLES = [
+  { name: '_row', desc: '1-based row counter' },
+  { name: '_index', desc: '0-based row counter' },
+  { name: '_count', desc: 'Total target rows' },
+  { name: '_name', desc: 'Current table name' },
+  { name: '_project', desc: 'Project name' },
+  { name: '_timestamp', desc: 'UTC ISO timestamp' },
+  { name: '_seed', desc: 'Project RNG seed' },
+];
+
 export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
   table,
   allProjectTables,
@@ -47,9 +63,141 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
   const [previewLoading, setPreviewLoading] = useState(false);
   const [previewError, setPreviewError] = useState<string | undefined>();
 
+  // Import / Export Modal state
+  const [isIoModalOpen, setIsIoModalOpen] = useState(false);
+  const [ioTab, setIoTab] = useState<'export' | 'import'>('export');
+  const [importText, setImportText] = useState('');
+  const [importError, setImportError] = useState<string | null>(null);
+  const [copied, setCopied] = useState(false);
+  const [showConfirmGuard, setShowConfirmGuard] = useState(false);
+
+  // Variable Picker Popover state
+  const [activeVarPickerIdx, setActiveVarPickerIdx] = useState<number | null>(null);
+  const [varSearchQuery, setVarSearchQuery] = useState('');
+  const varPickerRef = useRef<HTMLDivElement>(null);
+
+  useEffect(() => {
+    if (activeVarPickerIdx === null) return;
+    const handleClickOutside = (e: MouseEvent) => {
+      if (varPickerRef.current && !varPickerRef.current.contains(e.target as Node)) {
+        setActiveVarPickerIdx(null);
+      }
+    };
+    const handleKeyDown = (e: KeyboardEvent) => {
+      if (e.key === 'Escape') {
+        setActiveVarPickerIdx(null);
+      }
+    };
+    document.addEventListener('mousedown', handleClickOutside);
+    document.addEventListener('keydown', handleKeyDown);
+    return () => {
+      document.removeEventListener('mousedown', handleClickOutside);
+      document.removeEventListener('keydown', handleKeyDown);
+    };
+  }, [activeVarPickerIdx]);
+
+  const handleInsertVariable = (fieldIdx: number, varName: string) => {
+    const currentExpr = fields[fieldIdx]?.config?.expression || '';
+    handleConfigChange(fieldIdx, 'expression', currentExpr + varName);
+    setActiveVarPickerIdx(null);
+    setVarSearchQuery('');
+  };
+
   const isLocked = table.status === 'valid' && table.referencing_tables.length > 0;
   const isAggregate = table.archetype === 'aggregate';
 
+  const exportPayload = {
+    archetype: table.archetype,
+    format,
+    row_limit: rowLimit,
+    exclude_if: excludeIf || null,
+    fields,
+  };
+  const exportJsonString = JSON.stringify(exportPayload, null, 2);
+
+  const handleCopyExport = async () => {
+    try {
+      await navigator.clipboard.writeText(exportJsonString);
+      setCopied(true);
+      setTimeout(() => setCopied(false), 2000);
+    } catch (err) {
+      console.error('Failed to copy to clipboard', err);
+    }
+  };
+
+  const handleDownloadJson = () => {
+    const blob = new Blob([exportJsonString], { type: 'application/json' });
+    const url = URL.createObjectURL(blob);
+    const a = document.createElement('a');
+    a.href = url;
+    a.download = `${(table.slug || name).toLowerCase().replace(/\s+/g, '_')}_schema.json`;
+    a.click();
+    URL.revokeObjectURL(url);
+  };
+
+  const handleTriggerImport = () => {
+    if (isLocked) {
+      setImportError(
+        `Cannot alter schema: table is actively referenced by ${table.referencing_tables.join(', ')}.`
+      );
+      return;
+    }
+    if (!importText.trim()) {
+      setImportError('Please paste schema JSON.');
+      return;
+    }
+    try {
+      const parsed = JSON.parse(importText);
+      if (!Array.isArray(parsed) && (!parsed || !Array.isArray(parsed.fields))) {
+        setImportError("Invalid schema JSON. Expected '{ fields: [...] }' or field array.");
+        return;
+      }
+      setImportError(null);
+      setShowConfirmGuard(true);
+    } catch (err: any) {
+      setImportError(err.message || 'Invalid JSON format');
+    }
+  };
+
+  const handleConfirmImport = () => {
+    try {
+      const parsed = JSON.parse(importText);
+      let newFields: Field[] = [];
+
+      if (Array.isArray(parsed)) {
+        newFields = parsed;
+      } else if (parsed && Array.isArray(parsed.fields)) {
+        newFields = parsed.fields;
+        if (parsed.format && (parsed.format === 'csv' || parsed.format === 'parquet')) {
+          setFormat(parsed.format);
+        }
+        if (typeof parsed.row_limit === 'number') {
+          setRowLimit(parsed.row_limit);
+        }
+        if (typeof parsed.exclude_if === 'string' || parsed.exclude_if === null) {
+          setExcludeIf(parsed.exclude_if || '');
+        }
+      }
+
+      if (newFields.length === 0) {
+        setImportError('No columns/fields found in the imported JSON.');
+        setShowConfirmGuard(false);
+        return;
+      }
+
+      setFields(newFields);
+      setShowConfirmGuard(false);
+      setIsIoModalOpen(false);
+      setImportText('');
+      setSuccessMessage(
+        `Imported schema with ${newFields.length} columns. Click 'Save Draft' to persist.`
+      );
+    } catch (err: any) {
+      setImportError(err.message || 'Failed to parse schema JSON.');
+      setShowConfirmGuard(false);
+    }
+  };
+
   // Find all reference tables in project
   const refTables = allProjectTables.filter(
     (t) => t.archetype === 'reference'
@@ -155,17 +303,61 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
     });
   };
 
+  const handleChoicesChange = (index: number, text: string) => {
+    const parsed = text.split(',').map((s) => s.trim()).filter(Boolean);
+    setFields((prev) => {
+      const next = [...prev];
+      next[index] = {
+        ...next[index],
+        config: {
+          ...(next[index].config || {}),
+          choices: parsed,
+          choices_text: text,
+        },
+      };
+      return next;
+    });
+  };
+
+  const sanitizeFields = (fieldsToClean: Field[]): Field[] => {
+    return fieldsToClean.map((f) => {
+      const cfg = { ...(f.config || {}) };
+      if (typeof cfg.expression === 'string') cfg.expression = cfg.expression.trim();
+      if (typeof cfg.source_field === 'string') cfg.source_field = cfg.source_field.trim();
+      if (typeof cfg.ref_table === 'string') cfg.ref_table = cfg.ref_table.trim();
+      if (typeof cfg.collection_table === 'string') cfg.collection_table = cfg.collection_table.trim();
+      if (typeof cfg.key_col === 'string') cfg.key_col = cfg.key_col.trim();
+      if (typeof cfg.val_col === 'string') cfg.val_col = cfg.val_col.trim();
+      if (typeof cfg.mask === 'string') cfg.mask = cfg.mask.trim();
+      if (typeof cfg.start_date === 'string') cfg.start_date = cfg.start_date.trim();
+      if (typeof cfg.value === 'string') cfg.value = cfg.value.trim();
+      if (Array.isArray(cfg.choices)) cfg.choices = cfg.choices.map((c) => String(c).trim()).filter(Boolean);
+      if (typeof cfg.choices_text === 'string') cfg.choices_text = cfg.choices_text.trim();
+      return {
+        ...f,
+        name: f.name.trim(),
+        config: cfg,
+      };
+    });
+  };
+
   const handleSave = async () => {
     setSaving(true);
     setValidationError(null);
     setSuccessMessage(null);
+    const cleanName = name.trim();
+    const cleanExclude = excludeIf ? excludeIf.trim() : null;
+    const cleanFields = sanitizeFields(fields);
+    setName(cleanName);
+    setExcludeIf(cleanExclude || '');
+    setFields(cleanFields);
     try {
       const updated = await updateTable(table.id, {
-        name,
+        name: cleanName,
         format,
         row_limit: rowLimit,
-        exclude_if: excludeIf || null,
+        exclude_if: cleanExclude,
-        fields,
+        fields: cleanFields,
       });
       onTableUpdated(updated);
       setSuccessMessage('Table configuration saved.');
@@ -180,13 +372,19 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
     setValidating(true);
     setValidationError(null);
     setSuccessMessage(null);
+    const cleanName = name.trim();
+    const cleanExclude = excludeIf ? excludeIf.trim() : null;
+    const cleanFields = sanitizeFields(fields);
+    setName(cleanName);
+    setExcludeIf(cleanExclude || '');
+    setFields(cleanFields);
     try {
       await updateTable(table.id, {
-        name,
+        name: cleanName,
         format,
         row_limit: rowLimit,
-        exclude_if: excludeIf || null,
+        exclude_if: cleanExclude,
-        fields,
+        fields: cleanFields,
       });
       const res = await validateTable(table.id);
       if (res.success) {
@@ -276,6 +474,18 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
 
         {/* Global actions */}
         <div className="flex items-center space-x-3">
+          <button
+            onClick={() => {
+              setIsIoModalOpen(true);
+              setIoTab('export');
+              setImportError(null);
+            }}
+            className="flex items-center space-x-1.5 text-xs font-medium px-3.5 py-2 border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] hover:border-[#000000] dark:hover:border-[#FFFFFF] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+            title="Import or Export Schema JSON"
+          >
+            <ArrowsLeftRight size={14} />
+            <span>Import / Export</span>
+          </button>
           <button
             onClick={handleOpenPreview}
             className="flex items-center space-x-1.5 text-xs font-medium px-3.5 py-2 border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] hover:border-[#000000] dark:hover:border-[#FFFFFF] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
@@ -460,6 +670,7 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
                           <option value="uuid7">Type: uuid7</option>
                           <option value="choice">Type: choice</option>
                           <option value="collection">Type: collection</option>
+                          <option value="incremental">Type: incremental (1, 2, 3...)</option>
                         </>
                       )}
                     </select>
@@ -523,23 +734,159 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
                         <span className="font-medium text-[#000000] dark:text-[#FFFFFF]">
                           Python Expression (Safe Sandbox with f-strings, math, random):
                         </span>
-                        <div className="flex items-center space-x-1.5 text-[11px] font-mono text-[#4F6170] dark:text-[#ADBFCE]">
+
-                          <span>Upstream variables:</span>
+                        {/* Unified Variable Picker Button & Popover */}
-                          {upstreamColumns.length === 0 ? (
+                        <div className="relative">
-                            <span className="italic">none</span>
+                          <button
-                          ) : (
+                            type="button"
-                            upstreamColumns.map((col) => (
+                            disabled={isLocked || !isActive}
-                              <button
+                            onClick={(e) => {
-                                key={col}
+                              e.stopPropagation();
-                                onClick={() => {
+                              if (activeVarPickerIdx === idx) {
-                                  const currentExpr = field.config?.expression || '';
+                                setActiveVarPickerIdx(null);
-                                  handleConfigChange(idx, 'expression', currentExpr + col);
+                              } else {
-                                }}
+                                setActiveVarPickerIdx(idx);
-                                className="px-2 py-0.5 rounded bg-[#F4FAFF] dark:bg-[#161C22] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] hover:border-[#000000] dark:hover:border-[#FFFFFF] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+                                setVarSearchQuery('');
-                              >
+                              }
-                                {col}
+                            }}
-                              </button>
+                            className={`flex items-center space-x-1.5 text-xs font-mono font-medium px-2.5 py-1 rounded-[5px] border-[1.5px] transition-colors disabled:opacity-40 cursor-pointer ${
-                            ))
+                              activeVarPickerIdx === idx
+                                ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#EDF3EC] dark:bg-[#1b2b1e] text-[#346538] dark:text-[#88d48e]'
+                                : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#0E1317] hover:border-[#000000] dark:hover:border-[#FFFFFF] text-[#000000] dark:text-[#FFFFFF]'
+                            }`}
+                            title="Insert built-in variable or upstream column into expression"
+                          >
+                            <BracketsCurly size={13} weight="bold" />
+                            <span>{'{ }'} Insert Variable</span>
+                          </button>
+
+                          {activeVarPickerIdx === idx && (
+                            <div
+                              ref={varPickerRef}
+                              onClick={(e) => e.stopPropagation()}
+                              className="absolute right-0 top-full mt-1.5 w-80 max-h-[380px] overflow-y-auto z-40 bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[8px] shadow-xl p-3 space-y-3"
+                            >
+                              {/* Search Input */}
+                              <div className="relative">
+                                <MagnifyingGlass
+                                  size={13}
+                                  className="absolute left-2.5 top-1/2 -translate-y-1/2 text-[#4F6170] dark:text-[#ADBFCE]"
+                                />
+                                <input
+                                  type="text"
+                                  autoFocus
+                                  value={varSearchQuery}
+                                  onChange={(e) => setVarSearchQuery(e.target.value)}
+                                  placeholder="Filter variables..."
+                                  className="w-full pl-8 pr-2.5 py-1.5 text-xs font-mono border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[5px] bg-[#F4FAFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] focus:outline-none focus:border-[#000000] dark:focus:border-[#FFFFFF]"
+                                />
+                              </div>
+
+                              {/* Fundamental / Built-in Variables Section */}
+                              <div className="space-y-1.5">
+                                <div className="text-[10px] font-mono uppercase tracking-wider font-semibold text-[#4F6170] dark:text-[#ADBFCE] flex items-center justify-between">
+                                  <span>Fundamental Variables</span>
+                                  <span>
+                                    {
+                                      BUILTIN_METADATA_VARIABLES.filter(
+                                        (v) =>
+                                          v.name
+                                            .toLowerCase()
+                                            .includes(varSearchQuery.trim().toLowerCase()) ||
+                                          v.desc
+                                            .toLowerCase()
+                                            .includes(varSearchQuery.trim().toLowerCase())
+                                      ).length
+                                    }
+                                  </span>
+                                </div>
+                                {BUILTIN_METADATA_VARIABLES.filter(
+                                  (v) =>
+                                    v.name
+                                      .toLowerCase()
+                                      .includes(varSearchQuery.trim().toLowerCase()) ||
+                                    v.desc
+                                      .toLowerCase()
+                                      .includes(varSearchQuery.trim().toLowerCase())
+                                ).length === 0 ? (
+                                  <div className="text-[11px] font-mono text-[#4F6170] dark:text-[#ADBFCE] italic px-1">
+                                    No matching variables
+                                  </div>
+                                ) : (
+                                  <div className="grid grid-cols-1 gap-1">
+                                    {BUILTIN_METADATA_VARIABLES.filter(
+                                      (v) =>
+                                        v.name
+                                          .toLowerCase()
+                                          .includes(varSearchQuery.trim().toLowerCase()) ||
+                                        v.desc
+                                          .toLowerCase()
+                                          .includes(varSearchQuery.trim().toLowerCase())
+                                    ).map((v) => (
+                                      <button
+                                        key={v.name}
+                                        type="button"
+                                        onClick={() => handleInsertVariable(idx, v.name)}
+                                        className="flex items-center justify-between text-left px-2 py-1.5 rounded-[5px] bg-[#EDF3EC] dark:bg-[#1b2b1e] border border-[#DEE3E8] dark:border-[#4F6170] hover:border-[#000000] dark:hover:border-[#FFFFFF] transition-colors group cursor-pointer"
+                                      >
+                                        <span className="font-mono text-xs font-bold text-[#346538] dark:text-[#88d48e]">
+                                          {v.name}
+                                        </span>
+                                        <span className="font-mono text-[10px] text-[#4F6170] dark:text-[#ADBFCE] group-hover:text-[#000000] dark:group-hover:text-[#FFFFFF] transition-colors truncate max-w-[155px]">
+                                          {v.desc}
+                                        </span>
+                                      </button>
+                                    ))}
+                                  </div>
+                                )}
+                              </div>
+
+                              {/* Upstream Columns Section */}
+                              <div className="space-y-1.5 border-t-[1.5px] border-[#DEE3E8] dark:border-[#4F6170]/60 pt-2">
+                                <div className="text-[10px] font-mono uppercase tracking-wider font-semibold text-[#4F6170] dark:text-[#ADBFCE] flex items-center justify-between">
+                                  <span>Upstream Columns</span>
+                                  <span>
+                                    {
+                                      upstreamColumns.filter((col) =>
+                                        col
+                                          .toLowerCase()
+                                          .includes(varSearchQuery.trim().toLowerCase())
+                                      ).length
+                                    }
+                                  </span>
+                                </div>
+                                {upstreamColumns.filter((col) =>
+                                  col
+                                    .toLowerCase()
+                                    .includes(varSearchQuery.trim().toLowerCase())
+                                ).length === 0 ? (
+                                  <div className="text-[11px] font-mono text-[#4F6170] dark:text-[#ADBFCE] italic px-1">
+                                    {upstreamColumns.length === 0
+                                      ? 'No upstream columns defined before this column'
+                                      : 'No matching columns'}
+                                  </div>
+                                ) : (
+                                  <div className="flex flex-wrap gap-1.5 max-h-36 overflow-y-auto pr-1">
+                                    {upstreamColumns
+                                      .filter((col) =>
+                                        col
+                                          .toLowerCase()
+                                          .includes(varSearchQuery.trim().toLowerCase())
+                                      )
+                                      .map((col) => (
+                                        <button
+                                          key={col}
+                                          type="button"
+                                          onClick={() => handleInsertVariable(idx, col)}
+                                          className="px-2 py-1 text-xs font-mono font-medium rounded-[5px] bg-[#F4FAFF] dark:bg-[#161C22] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] hover:border-[#000000] dark:hover:border-[#FFFFFF] text-[#000000] dark:text-[#FFFFFF] transition-colors hover:bg-[#FFFFFF] dark:hover:bg-[#0E1317] cursor-pointer"
+                                        >
+                                          {col}
+                                        </button>
+                                      ))}
+                                  </div>
+                                )}
+                              </div>
+                            </div>
                           )}
                         </div>
                       </div>
@@ -754,14 +1101,8 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
                               </label>
                               <input
                                 type="text"
-                                value={(field.config?.choices || []).join(', ')}
+                                value={field.config?.choices_text ?? (field.config?.choices || []).join(', ')}
-                                onChange={(e) =>
+                                onChange={(e) => handleChoicesChange(idx, e.target.value)}
-                                  handleConfigChange(
-                                    idx,
-                                    'choices',
-                                    e.target.value.split(',').map((s) => s.trim()).filter(Boolean)
-                                  )
-                                }
                                 placeholder="e.g. REG_JKT, REG_BDG, REG_SBY"
                                 className="w-full px-2.5 py-1.5 font-mono text-xs border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[5px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] focus:outline-none focus:border-[#000000] dark:focus:border-[#FFFFFF]"
                               />
@@ -883,14 +1224,8 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
                           </label>
                           <input
                             type="text"
-                            value={(field.config?.choices || []).join(', ')}
+                            value={field.config?.choices_text ?? (field.config?.choices || []).join(', ')}
-                            onChange={(e) =>
+                            onChange={(e) => handleChoicesChange(idx, e.target.value)}
-                              handleConfigChange(
-                                idx,
-                                'choices',
-                                e.target.value.split(',').map((s) => s.trim()).filter(Boolean)
-                              )
-                            }
                             placeholder="e.g. Cash, Card, Online"
                             className="w-full px-2.5 py-1.5 font-mono text-xs border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[5px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] focus:outline-none focus:border-[#000000] dark:focus:border-[#FFFFFF]"
                           />
@@ -923,6 +1258,36 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
                         </div>
                       )}
 
+                      {field.type === 'incremental' && (
+                        <>
+                          <div>
+                            <label className="block text-[#4F6170] dark:text-[#ADBFCE] text-[11px] mb-1 font-medium">Start Index</label>
+                            <input
+                              type="number"
+                              value={field.config?.start ?? 1}
+                              onChange={(e) =>
+                                handleConfigChange(idx, 'start', parseInt(e.target.value) || 0)
+                              }
+                              className="w-full px-2.5 py-1.5 font-mono text-xs border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[5px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] focus:outline-none"
+                            />
+                          </div>
+                          <div>
+                            <label className="block text-[#4F6170] dark:text-[#ADBFCE] text-[11px] mb-1 font-medium">Step Increment</label>
+                            <input
+                              type="number"
+                              value={field.config?.step ?? 1}
+                              onChange={(e) =>
+                                handleConfigChange(idx, 'step', parseInt(e.target.value) || 1)
+                              }
+                              className="w-full px-2.5 py-1.5 font-mono text-xs border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[5px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] focus:outline-none"
+                            />
+                          </div>
+                          <div className="col-span-2 text-[#4F6170] dark:text-[#ADBFCE] text-xs font-mono py-1 flex items-center">
+                            Sequence: {field.config?.start ?? 1}, {(field.config?.start ?? 1) + (field.config?.step ?? 1)}, {(field.config?.start ?? 1) + (field.config?.step ?? 1) * 2}...
+                          </div>
+                        </>
+                      )}
+
                       {field.type === 'uuid7' && (
                         <div className="col-span-3 text-[#4F6170] dark:text-[#ADBFCE] text-xs font-mono py-1">
                           Generates RFC 9562 time-ordered 128-bit UUIDv7 strings.
@@ -930,15 +1295,15 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
                       )}
 
                       {field.type === 'string' && (
-                        <div className="col-span-2">
+                        <div className="col-span-3">
                           <label className="block text-[#4F6170] dark:text-[#ADBFCE] text-[11px] mb-1 font-medium">
-                            Mask Pattern (# = digit, ? = uppercase, * = alphanumeric)
+                            Mask Pattern (# = digit, ? = uppercase, @ = lowercase, % = hex, * = alphanumeric, \ = escape)
                           </label>
                           <input
                             type="text"
                             value={field.config?.mask || ''}
                             onChange={(e) => handleConfigChange(idx, 'mask', e.target.value)}
-                            placeholder="e.g. USR-####"
+                            placeholder="e.g. USR-####, MAC-\#%%-\#%%, 0x%%%%"
                             className="w-full px-2.5 py-1.5 font-mono text-xs border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[5px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] focus:outline-none focus:border-[#000000] dark:focus:border-[#FFFFFF]"
                           />
                         </div>
@@ -975,6 +1340,167 @@ export const TableSchemaEditor: React.FC<TableSchemaEditorProps> = ({
         loading={previewLoading}
         error={previewError}
       />
+
+      {/* Import / Export Modal */}
+      {isIoModalOpen && (
+        <div className="fixed inset-0 z-50 flex items-center justify-center bg-black/40 backdrop-blur-xs p-4">
+          <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] w-full max-w-2xl p-6 space-y-4 shadow-xl">
+            {/* Header: Horizontal tabs on top-left, Close button on right */}
+            <div className="flex items-center justify-between border-b border-[#DEE3E8] dark:border-[#4F6170] pb-3">
+              <div className="flex items-center space-x-2">
+                <button
+                  type="button"
+                  onClick={() => {
+                    setIoTab('export');
+                    setImportError(null);
+                  }}
+                  className={`px-3 py-1.5 text-xs font-mono rounded-[6px] border transition-colors ${
+                    ioTab === 'export'
+                      ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000]'
+                      : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#0E1317]'
+                  }`}
+                >
+                  Export
+                </button>
+                <button
+                  type="button"
+                  onClick={() => {
+                    setIoTab('import');
+                    setImportError(null);
+                  }}
+                  className={`px-3 py-1.5 text-xs font-mono rounded-[6px] border transition-colors ${
+                    ioTab === 'import'
+                      ? 'border-[#000000] dark:border-[#FFFFFF] bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000]'
+                      : 'border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#0E1317]'
+                  }`}
+                >
+                  Import
+                </button>
+              </div>
+
+              <button
+                type="button"
+                onClick={() => {
+                  setIsIoModalOpen(false);
+                  setShowConfirmGuard(false);
+                  setImportError(null);
+                }}
+                className="p-1.5 rounded-[6px] border border-[#DEE3E8] dark:border-[#4F6170] text-[#4F6170] dark:text-[#ADBFCE] hover:text-[#000000] dark:hover:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] transition-colors"
+              >
+                <X size={15} />
+              </button>
+            </div>
+
+            {/* Tab 1: Export */}
+            {ioTab === 'export' && (
+              <div className="space-y-4">
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono">
+                  Table schema definition (name excluded). Suitable for version control and migration.
+                </p>
+                <textarea
+                  readOnly
+                  rows={12}
+                  value={exportJsonString}
+                  className="w-full px-3 py-2 text-xs font-mono border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] bg-[#F4FAFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none select-all"
+                />
+                <div className="flex items-center justify-end space-x-3 pt-2">
+                  <button
+                    type="button"
+                    onClick={handleCopyExport}
+                    className="flex items-center space-x-1.5 text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] transition-colors"
+                  >
+                    {copied ? <Check size={14} weight="bold" /> : <Clipboard size={14} />}
+                    <span>{copied ? 'Copied!' : 'Copy to Clipboard'}</span>
+                  </button>
+                  <button
+                    type="button"
+                    onClick={handleDownloadJson}
+                    className="flex items-center space-x-1.5 text-xs font-medium px-4 py-2 bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000] rounded-[6px] hover:bg-[#525456] dark:hover:bg-[#DEE3E8] transition-all"
+                  >
+                    <DownloadSimple size={14} weight="bold" />
+                    <span>Download .json</span>
+                  </button>
+                </div>
+              </div>
+            )}
+
+            {/* Tab 2: Import */}
+            {ioTab === 'import' && (
+              <div className="space-y-4">
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono">
+                  Paste valid JSON specification. Current table name remains preserved.
+                </p>
+                <textarea
+                  rows={12}
+                  value={importText}
+                  disabled={isLocked}
+                  onChange={(e) => setImportText(e.target.value)}
+                  placeholder='{ "archetype": "record", "format": "csv", "row_limit": 100000, "fields": [...] }'
+                  className="w-full px-3 py-2 text-xs font-mono border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] bg-[#FFFFFF] dark:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF] rounded-[6px] focus:outline-none focus:border-[#000000] dark:focus:border-[#ADBFCE] disabled:opacity-50"
+                />
+
+                {importError && (
+                  <div className="p-3 bg-[#FDEBEC] dark:bg-[#321214] border border-[#DEE3E8] dark:border-[#9F2F2D] rounded-[6px] text-xs font-mono text-[#9F2F2D] dark:text-[#FDEBEC]">
+                    {importError}
+                  </div>
+                )}
+
+                <div className="flex items-center justify-end space-x-3 pt-2">
+                  <button
+                    type="button"
+                    onClick={() => {
+                      setIsIoModalOpen(false);
+                      setImportText('');
+                      setImportError(null);
+                    }}
+                    className="text-xs font-medium px-3.5 py-2 border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] text-[#000000] dark:text-[#FFFFFF]"
+                  >
+                    Cancel
+                  </button>
+                  <button
+                    type="button"
+                    disabled={isLocked || !importText.trim()}
+                    onClick={handleTriggerImport}
+                    className="text-xs font-medium px-4 py-2 bg-[#000000] dark:bg-[#FFFFFF] text-[#FFFFFF] dark:text-[#000000] rounded-[6px] hover:bg-[#525456] dark:hover:bg-[#DEE3E8] disabled:opacity-40 transition-all"
+                  >
+                    Import
+                  </button>
+                </div>
+              </div>
+            )}
+          </div>
+
+          {/* Confirmation Guard Dialog */}
+          {showConfirmGuard && (
+            <div className="fixed inset-0 z-60 flex items-center justify-center bg-[#000000]/60 backdrop-blur-xs p-4">
+              <div className="bg-[#FFFFFF] dark:bg-[#0E1317] border-[1.5px] border-[#DEE3E8] dark:border-[#4F6170] rounded-[10px] max-w-md w-full p-6 shadow-2xl space-y-4">
+                <h3 className="text-base font-semibold text-[#000000] dark:text-[#FFFFFF]">
+                  Confirm Schema Overwrite
+                </h3>
+                <p className="text-xs text-[#4F6170] dark:text-[#ADBFCE] font-mono leading-relaxed">
+                  Are you sure you want to import? This will overwrite the current columns/data in this table.
+                </p>
+                <div className="flex items-center justify-end space-x-3 pt-3 border-t border-[#DEE3E8] dark:border-[#4F6170]">
+                  <button
+                    type="button"
+                    onClick={() => setShowConfirmGuard(false)}
+                    className="px-3.5 py-2 text-xs font-medium border border-[#DEE3E8] dark:border-[#4F6170] rounded-[6px] bg-[#FFFFFF] dark:bg-[#0E1317] text-[#000000] dark:text-[#FFFFFF] hover:bg-[#F4FAFF] dark:hover:bg-[#161C22] transition-colors"
+                  >
+                    Cancel
+                  </button>
+                  <button
+                    type="button"
+                    onClick={handleConfirmImport}
+                    className="px-4 py-2 text-xs font-medium bg-[#9F2F2D] text-[#FFFFFF] rounded-[6px] hover:bg-[#7D2523] transition-colors"
+                  >
+                    Overwrite & Import
+                  </button>
+                </div>
+              </div>
+            </div>
+          )}
+        </div>
+      )}
     </div>
   );
 };

frontend/src/types/index.ts

@@ -2,7 +2,7 @@ export type TableArchetype = 'record' | 'aggregate' | 'reference' | 'collection'
 export type TableStatus = 'draft' | 'valid';
 export type OutputFormat = 'csv' | 'parquet';
 export type ColumnRole = 'aggregated' | 'randomized' | 'function' | 'reference' | 'static';
-export type ColumnType = 'string' | 'integer' | 'float' | 'datetime' | 'boolean' | 'uuid7' | 'choice' | 'collection';
+export type ColumnType = 'string' | 'integer' | 'float' | 'datetime' | 'boolean' | 'uuid7' | 'choice' | 'collection' | 'incremental';
 
 export interface FieldConfig {
   expression?: string;
@@ -11,6 +11,8 @@ export interface FieldConfig {
   collection_table?: string;
   key_col?: string;
   val_col?: string;
+  start?: number;
+  step?: number;
   min?: number;
   max?: number;
   precision?: number;
@@ -18,6 +20,7 @@ export interface FieldConfig {
   mean?: number;
   stddev?: number;
   choices?: string[];
+  choices_text?: string;
   weights?: number[];
   start_date?: string;
   days?: number;