feat: minor ui changes

.env.example

@@ -0,0 +1,5 @@
+LLM_API_KEY=your_api_key_here
+LLM_BASE_URL=https://api.openai.com/v1
+LLM_MODEL=your_llm_model_here
+DATABASE_URL=postgresql://user:password@localhost:5432/silksoup
+OUTPUT_DIR=output

.gitignore

@@ -3,4 +3,5 @@ dev/
 __pycache__/
 *.pyc
 sqlite.db
-.env
+.env
+output/temp/

Dockerfile

@@ -0,0 +1,18 @@
+FROM python:3.12-slim
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential curl libxml2-dev libxslt1-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+RUN mkdir -p output
+
+EXPOSE 8000
+
+CMD ["uvicorn", "web.app:app", "--host", "0.0.0.0", "--port", "8000"]

README.md

@@ -0,0 +1,122 @@
+# Silksoup
+
+> Turn online web novels and light novels into clean, beautifully formatted EPUB e-books with a single click or command.
+
+Silksoup scrapes web novel index pages, uses an OpenAI-compatible LLM to detect volumes and chapters, cleans out site ads and junk, downloads illustrations, and packages everything into an offline `.epub` file ready for your e-reader, phone, or tablet.
+
+---
+
+## Key Features
+
+- **Dual Modes:** Use the visual **Web Dashboard** or the terminal **CLI tool**—both share the same engine.
+- **Smart Chapter Extraction:** Uses AI to understand complex table of contents, volumes, and reading order.
+- **Clean Reading Experience:** Strips navigation links, donation banners, and website watermarks while preserving important translator notes, formatting, and character dialogue.
+- **Image & Illustration Support:** Downloads cover art and embedded chapter illustrations (supports WEBP, JPEG, PNG, GIF) with anti-hotlinking headers.
+- **Smart Caching (PostgreSQL & SQLite):** Stores downloaded chapters so re-runs and updates don't re-download what you already have.
+- **Self-Hostable & Docker Ready:** Run it locally or host it on a VPS behind WireGuard / VPN.
+
+---
+
+## Quick Start (Web App)
+
+### Option A: Using Docker (Recommended for Servers / VPS)
+
+1. **Clone the repository:**
+   ```bash
+   git clone https://github.com/your-username/silksoup.git
+   cd silksoup
+   ```
+
+2. **Set up your environment variables:**
+   ```bash
+   cp .env.example .env
+   ```
+   Open `.env` and configure your OpenAI-compatible LLM:
+   ```env
+   LLM_API_KEY=your_api_key_here
+   LLM_BASE_URL=https://api.openai.com/v1
+   LLM_MODEL=ds/deepseek-v4-flash
+
+   # Optional: Connect to your PostgreSQL database (leave blank to use local SQLite)
+   DATABASE_URL=postgresql://user:password@localhost:5432/silksoup
+   ```
+
+3. **Start the service:**
+   ```bash
+   docker compose up -d --build
+   ```
+
+4. **Open your browser:**
+   Go to `http://localhost:8000` (or `http://YOUR_SERVER_IP:8000`).
+
+---
+
+### Option B: Running Locally with Python
+
+1. **Install dependencies:**
+   ```bash
+   pip install -r requirements.txt
+   # or with uv:
+   uv pip install -r requirements.txt
+   ```
+
+2. **Configure `.env`** (see step 2 above).
+
+3. **Start the web server:**
+   ```bash
+   python server.py
+   ```
+   Open `http://localhost:8000` in your browser.
+
+---
+
+## Web Dashboard Guide
+
+1. **Enter Novel URL:** Paste the main index or table-of-contents link of the novel into the input box.
+2. **Start Scraping:** Click **"Start Scraping -> EPUB"**.
+3. **Live Progress:** Watch real-time progress as chapters are fetched, cleaned, and compiled into the EPUB.
+4. **Download & Manage:** Once complete, click **"Download EPUB"** or manage your compiled books directly from the built-in **Library**.
+
+---
+
+## CLI Usage
+
+If you prefer working from the terminal, run `main.py` directly:
+
+### CLI Syntax:
+```bash
+python main.py <url> [-o OUTPUT] [--refresh [{none,llm,all}]]
+```
+
+### Flags & Options:
+
+| Option / Flag | Type | Description |
+|---|---|---|
+| `url` | *Positional (Required)* | The main index / table-of-contents URL of the novel. |
+| `-o`, `--output` | *Optional* | Output directory for the compiled EPUB file (default: `./output`). |
+| `--refresh` | *Optional (Choice)* | Cache refresh mode: `none` (default, full cache), `llm` (reuse HTML, re-run AI), or `all` (full scrape anew). Providing `--refresh` without an argument defaults to `all`. |
+| `-h`, `--help` | *Optional (Flag)* | Show command-line help message and exit. |
+
+---
+
+## Where Are My EPUB Files?
+
+All compiled `.epub` files are saved directly inside the `output/` directory in the project folder:
+- **Host Path:** `./output/<novel-title>.epub`
+- **Docker Path:** `/app/output/<novel-title>.epub`
+
+You can move these files to your e-reader (Kindle, Kobo, Boox) or integrate the folder with media servers like **Calibre-Web**, **Kavita**, or **Audiobookshelf**.
+
+---
+
+## Configuration Reference
+
+All settings can be configured inside your `.env` file:
+
+| Variable | Description | Default |
+|---|---|---|
+| `LLM_API_KEY` | **Required.** Your API key for the OpenAI-compatible LLM. | *None* |
+| `LLM_BASE_URL` | **Required.** Base URL of any OpenAI-compatible API endpoint. | *None* |
+| `LLM_MODEL` | **Required.** Model identifier used for extraction & cleaning. | *None* |
+| `DATABASE_URL` | PostgreSQL connection string. If omitted, uses local `sqlite.db`. | *None (SQLite)* |
+| `OUTPUT_DIR` | Directory where compiled EPUBs are stored. | `output` |

docker-compose.yml

@@ -0,0 +1,17 @@
+services:
+  app:
+    build: .
+    container_name: silksoup
+    ports:
+      - "127.0.0.1:8000:8000"
+    env_file:
+      - .env
+    environment:
+      - DATABASE_URL=${DATABASE_URL:-}
+      - LLM_API_KEY=${LLM_API_KEY}
+      - LLM_BASE_URL=${LLM_BASE_URL}
+      - LLM_MODEL=${LLM_MODEL}
+      - OUTPUT_DIR=/app/output
+    volumes:
+      - ./output:/app/output
+    restart: unless-stopped

main.py

@@ -1,18 +1,54 @@
+import argparse
+from pathlib import Path
 from src.database import init_db
 from src.metadata import extract_metadata_json
 from src.parse import cache_hit
 from src.clean_html import clean_novel_html
 from src.chapter import clean_chapters
+from src.epub_creator import create_epub
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Scrape light novel and compile EPUB."
+    )
+    parser.add_argument(
+        "url",
+        type=str,
+        help="URL of the novel table of contents / index page",
+    )
+    parser.add_argument(
+        "-o",
+        "--output",
+        type=str,
+        default=str(Path(__file__).resolve().parent / "output"),
+        help="Output directory for the compiled EPUB (default: ./output)",
+    )
+    parser.add_argument(
+        "--refresh",
+        dest="refresh_mode",
+        choices=["none", "llm", "all"],
+        nargs="?",
+        const="all",
+        default="none",
+        help="Cache refresh mode: 'none' (use full cache), 'llm' (reuse HTML, re-run AI), 'all' (full fresh scrape)",
+    )
+    return parser.parse_args()
 
 
 def main() -> None:
+    args = parse_args()
     init_db()
 
-    page_url = "https://hikkicave.site/story/i-became-friends-with-the-second-cutest-girl-in-my-class-wn/"
-
-    html_content = clean_novel_html(cache_hit(page_url))
-    metadata = extract_metadata_json(html_content, page_url)
-    clean_chapters(metadata)
+    html_content = clean_novel_html(
+        cache_hit(args.url, refresh_mode=args.refresh_mode),
+        base_url=args.url,
+    )
+    metadata = extract_metadata_json(
+        html_content, args.url, refresh_mode=args.refresh_mode
+    )
+    clean_chapters(metadata, refresh_mode=args.refresh_mode)
+    create_epub(metadata, output_dir=args.output)
 
 
 if __name__ == "__main__":

requirements.txt

@@ -0,0 +1,14 @@
+beautifulsoup4==4.15.0
+lxml==6.1.2
+curl-cffi==0.16.1
+ebooklib==0.20
+Pillow==12.3.0
+tqdm==4.70.0
+openai==3.3.1
+python-dotenv==1.2.3
+fastapi==0.115.0
+uvicorn[standard]==0.30.6
+jinja2==3.1.4
+psycopg[binary]==3.2.3
+python-multipart==0.0.9
+httpx==0.28.1

server.py

@@ -0,0 +1,4 @@
+import uvicorn
+
+if __name__ == "__main__":
+    uvicorn.run("web.app:app", host="0.0.0.0", port=8000, reload=False)

src/ape.py

@@ -0,0 +1,236 @@
+import json
+import re
+from bs4 import BeautifulSoup
+from .database import db_execute, db_fetch_one
+from .llm import generate_json, generate_text
+from .prompt import (
+    APE_SYSTEM_PROMPT,
+    CHAPTER_SUMMARY_SYSTEM_PROMPT,
+    LORE_EXTRACTION_SYSTEM_PROMPT,
+)
+
+
+def get_or_init_novel_lore(novel_url: str, metadata: dict, initial_sample_html: str = "") -> dict:
+    row = db_fetch_one("SELECT lore_json FROM novel_lore WHERE novel_url = ?", (novel_url,))
+    if row is not None and row[0]:
+        try:
+            return json.loads(row[0])
+        except Exception:
+            pass
+
+    series_meta = metadata.get("metadata", {})
+    description = series_meta.get("description") or ""
+    title = series_meta.get("title") or "Novel"
+
+    soup = BeautifulSoup(initial_sample_html, "lxml")
+    sample_text = soup.get_text(separator=" ", strip=True)[:4000]
+
+    input_text = f"Title: {title}\nDescription: {description}\n\nSample Chapter Content:\n{sample_text}"
+    try:
+        raw_json = generate_json(LORE_EXTRACTION_SYSTEM_PROMPT, input_text)
+        lore = json.loads(raw_json)
+    except Exception:
+        lore = {
+            "characters": [],
+            "terms": {},
+            "rules": {"keep_honorifics": False, "pov": "Third person limited"},
+        }
+
+    lore_str = json.dumps(lore, ensure_ascii=False)
+    db_execute(
+        "INSERT INTO novel_lore (novel_url, lore_json, rolling_summary_json) VALUES (?, ?, ?) ON CONFLICT(novel_url) DO UPDATE SET lore_json=excluded.lore_json, updated_at=CURRENT_TIMESTAMP",
+        (novel_url, lore_str, json.dumps([])),
+    )
+    return lore
+
+
+def get_rolling_summary(novel_url: str) -> list:
+    row = db_fetch_one("SELECT rolling_summary_json FROM novel_lore WHERE novel_url = ?", (novel_url,))
+    if row is not None and row[0]:
+        try:
+            return json.loads(row[0])
+        except Exception:
+            return []
+    return []
+
+
+def update_rolling_summary(novel_url: str, chapter_name: str, chapter_html: str) -> None:
+    soup = BeautifulSoup(chapter_html, "lxml")
+    plain_text = soup.get_text(separator=" ", strip=True)
+    if not plain_text:
+        return
+
+    sample = plain_text[:3500]
+    try:
+        summary_text = generate_text(CHAPTER_SUMMARY_SYSTEM_PROMPT, sample).strip()
+    except Exception:
+        summary_text = f"Events occurred in {chapter_name}."
+
+    existing = get_rolling_summary(novel_url)
+    existing.append({"title": chapter_name, "summary": summary_text})
+    recent = existing[-2:]
+
+    db_execute(
+        "INSERT INTO novel_lore (novel_url, rolling_summary_json) VALUES (?, ?) ON CONFLICT(novel_url) DO UPDATE SET rolling_summary_json=excluded.rolling_summary_json, updated_at=CURRENT_TIMESTAMP",
+        (novel_url, json.dumps(recent, ensure_ascii=False)),
+    )
+
+
+def strip_duplicate_chapter_title(html: str, chapter_title: str) -> str:
+    if not html or not html.strip():
+        return ""
+    soup = BeautifulSoup(html, "lxml")
+    body = soup.body if soup.body else soup
+
+    # Clean title for comparison
+    clean_target = re.sub(r"[^\w\s]", "", chapter_title.lower()).strip()
+    target_words = set(clean_target.split())
+
+    # Check top headings or paragraphs
+    for tag in list(body.find_all(["h1", "h2", "h3", "p", "div"], recursive=False))[:3]:
+        tag_text = tag.get_text(strip=True)
+        clean_tag = re.sub(r"[^\w\s]", "", tag_text.lower()).strip()
+        tag_words = set(clean_tag.split())
+
+        # Exact match or high overlap with chapter title / "chapter X"
+        if clean_target and (clean_tag == clean_target or (len(target_words) >= 2 and target_words.issubset(tag_words))):
+            tag.decompose()
+            continue
+        if re.match(r"^(chapter|episode|prologue|epilogue)\s*\d*", clean_tag, re.I) and len(clean_tag.split()) <= 4:
+            tag.decompose()
+            continue
+
+    return body.decode_contents().strip()
+
+
+def chunk_chapter_text(html: str, target_words: int = 800) -> list[str]:
+    if not html or not html.strip():
+        return []
+
+    soup = BeautifulSoup(html, "lxml")
+    body = soup.body if soup.body else soup
+
+    elements = list(body.find_all(["p", "hr", "img", "blockquote", "aside"], recursive=False))
+    if not elements:
+        # Fallback: split by paragraph or line breaks
+        paragraphs = [p.strip() for p in html.split("\n") if p.strip()]
+        chunks = []
+        curr = []
+        curr_words = 0
+        for p in paragraphs:
+            w_count = len(p.split())
+            if curr_words + w_count > target_words and curr:
+                chunks.append("\n".join(curr))
+                curr = [p]
+                curr_words = w_count
+            else:
+                curr.append(p)
+                curr_words += w_count
+        if curr:
+            chunks.append("\n".join(curr))
+        return chunks
+
+    chunks = []
+    curr_elements = []
+    curr_word_count = 0
+
+    for el in elements:
+        text = el.get_text(strip=True)
+        w_count = len(text.split())
+        if curr_word_count + w_count > target_words and curr_elements:
+            chunk_html = "".join(str(e) for e in curr_elements)
+            chunks.append(chunk_html)
+            curr_elements = [el]
+            curr_word_count = w_count
+        else:
+            curr_elements.append(el)
+            curr_word_count += w_count
+
+    if curr_elements:
+        chunk_html = "".join(str(e) for e in curr_elements)
+        chunks.append(chunk_html)
+
+    return chunks
+
+
+def filter_glossary_for_chunk(global_lore: dict, chunk_text: str) -> dict:
+    if not global_lore:
+        return {}
+
+    lowered_chunk = chunk_text.lower()
+    filtered_chars = []
+    for char in global_lore.get("characters", []):
+        names = [char.get("standard_name", ""), char.get("raw_name", "")] + char.get("mtl_variants", [])
+        if any(n and n.lower() in lowered_chunk for n in names):
+            filtered_chars.append(char)
+
+    filtered_terms = {}
+    for term, trans in global_lore.get("terms", {}).items():
+        if (term and term.lower() in lowered_chunk) or (trans and trans.lower() in lowered_chunk):
+            filtered_terms[term] = trans
+
+    # If no matches, retain top character or default to keep context active
+    if not filtered_chars and global_lore.get("characters"):
+        filtered_chars = global_lore["characters"][:2]
+
+    return {
+        "characters": filtered_chars,
+        "terms": filtered_terms,
+        "rules": global_lore.get("rules", {}),
+    }
+
+
+def clean_llm_html_output(raw_output: str) -> str:
+    cleaned = re.sub(r"^```(?:html)?\s*", "", raw_output.strip(), flags=re.I)
+    cleaned = re.sub(r"\s*```$", "", cleaned)
+    # Ensure any <aside class="tl-note"> is converted to <blockquote>
+    cleaned = re.sub(r"<aside\s+class=[\"']tl-note[\"']>(.*?)</aside>", r"<blockquote class=\"tl-note\"><p><strong>TL Note:</strong> \1</p></blockquote>", cleaned, flags=re.DOTALL | re.I)
+    return cleaned.strip()
+
+
+def fix_chapter(
+    raw_cleaned_html: str,
+    chapter_name: str,
+    novel_url: str,
+    global_lore: dict,
+) -> str:
+    # 1. Deduplicate title from start of body
+    html_without_title = strip_duplicate_chapter_title(raw_cleaned_html, chapter_name)
+
+    # 2. Chunk text
+    chunks = chunk_chapter_text(html_without_title, target_words=800)
+    if not chunks:
+        chunks = [html_without_title] if html_without_title else ["<p></p>"]
+
+    # 3. Format rolling scene context
+    rolling = get_rolling_summary(novel_url)
+    if rolling:
+        context_lines = [f"- {r.get('title', 'Previous')}: {r.get('summary', '')}" for r in rolling]
+        scene_context_str = "Recent Scene Context:\n" + "\n".join(context_lines)
+    else:
+        scene_context_str = f"Scene Context: Opening / early chapter ({chapter_name})."
+
+    fixed_chunks = []
+    for chunk in chunks:
+        active_glossary = filter_glossary_for_chunk(global_lore, chunk)
+        user_prompt = (
+            f"### Active Glossary & Lore\n```json\n{json.dumps(active_glossary, ensure_ascii=False, indent=2)}\n```\n\n"
+            f"### Scene Context\n{scene_context_str}\n\n"
+            f"### Raw MTL Text to Process\n{chunk}"
+        )
+
+        try:
+            polished = generate_text(APE_SYSTEM_PROMPT, user_prompt)
+            fixed_chunk = clean_llm_html_output(polished)
+        except Exception:
+            # Fallback to original chunk if LLM fails on a segment
+            fixed_chunk = clean_llm_html_output(chunk)
+
+        fixed_chunks.append(fixed_chunk)
+
+    final_fixed_html = "\n".join(fixed_chunks)
+
+    # 4. Update rolling summary for next chapters
+    update_rolling_summary(novel_url, chapter_name, final_fixed_html)
+
+    return final_fixed_html

src/chapter.py

@@ -1,15 +1,10 @@
-import sqlite3
-from google import genai
-from google.genai import types
-from dotenv import load_dotenv
-from .database import DB_PATH
+from tqdm import tqdm
+from .database import db_execute, db_fetch_one
 from .parse import robust_fetch
 from .clean_html import clean_novel_html
 from .prompt import HTML_CHAPTER_SANITIZER
-
-
-load_dotenv()
-client = genai.Client()
+from .llm import generate_text
+from .ape import get_or_init_novel_lore, fix_chapter
 
 
 def iter_chapters(metadata: dict):
@@ -24,80 +19,178 @@ def iter_chapters(metadata: dict):
 
 
 def sanitize_chapter_html(html: str) -> str:
-    response = client.models.generate_content(
-        model="gemini-3.6-flash",
-        contents=html,
-        config=types.GenerateContentConfig(
-            system_instruction=HTML_CHAPTER_SANITIZER,
-            temperature=0.0,
-            response_mime_type="text/plain",
-            tools=None,
-        ),
-    )
-    return response.text.strip()
+    return generate_text(HTML_CHAPTER_SANITIZER, html)
 
 
-def cache_chapter_hit(url: str) -> str:
-    with sqlite3.connect(DB_PATH) as conn:
-        cursor = conn.cursor()
-        cursor.execute("SELECT html_text FROM chapter_page WHERE url = ?", (url,))
-        row = cursor.fetchone()
+def cache_chapter_hit(url: str, refresh_mode: str = "none") -> str:
+    if isinstance(refresh_mode, bool):
+        refresh_mode = "all" if refresh_mode else "none"
+
+    if refresh_mode not in ("all", "all_refresh"):
+        row = db_fetch_one("SELECT html_text FROM chapter_page WHERE url = ?", (url,))
         if row is not None and row[0] is not None:
-            print(f"Chapter cache hit: {url}")
+            tqdm.write(f"Chapter cache hit: {url}")
             return str(row[0])
 
     status_code, html = robust_fetch(url)
     if status_code != 200:
         raise RuntimeError(f"Request to {url} failed with status code {status_code}")
 
-    with sqlite3.connect(DB_PATH) as conn:
-        conn.execute(
-            "INSERT OR REPLACE INTO chapter_page (url, status_code, html_text) VALUES (?, ?, ?)",
-            (url, status_code, html),
-        )
-        conn.commit()
+    db_execute(
+        "INSERT INTO chapter_page (url, status_code, html_text) VALUES (?, ?, ?) ON CONFLICT(url) DO UPDATE SET status_code=excluded.status_code, html_text=excluded.html_text, fetched_at=CURRENT_TIMESTAMP",
+        (url, status_code, html),
+    )
 
     return html
 
 
-def cache_clean_hit(url: str) -> str:
-    with sqlite3.connect(DB_PATH) as conn:
-        cursor = conn.cursor()
-        cursor.execute("SELECT clean_html_text FROM chapter_page WHERE url = ?", (url,))
-        row = cursor.fetchone()
+def cache_clean_hit(url: str, refresh_mode: str = "none") -> str:
+    if isinstance(refresh_mode, bool):
+        refresh_mode = "all" if refresh_mode else "none"
+
+    if refresh_mode == "none":
+        row = db_fetch_one("SELECT clean_html_text FROM chapter_page WHERE url = ?", (url,))
         if row is not None and row[0] is not None:
-            print(f"Clean HTML cache hit: {url}")
+            tqdm.write(f"Clean HTML cache hit: {url}")
             return str(row[0])
 
-    raw_html = cache_chapter_hit(url)
+    raw_html = cache_chapter_hit(url, refresh_mode=refresh_mode)
     pre_cleaned = clean_novel_html(raw_html, base_url=url)
     sanitized = sanitize_chapter_html(pre_cleaned)
 
-    with sqlite3.connect(DB_PATH) as conn:
-        conn.execute(
-            "UPDATE chapter_page SET clean_html_text = ? WHERE url = ?",
-            (sanitized, url),
-        )
-        conn.commit()
+    db_execute(
+        "UPDATE chapter_page SET clean_html_text = ? WHERE url = ?",
+        (sanitized, url),
+    )
 
     return sanitized
 
 
-def scrape_chapters(metadata: dict) -> None:
-    for chapter in iter_chapters(metadata):
-        print(f"[{chapter['volume']}] {chapter['name']} -> {chapter['url']}")
-        try:
-            cache_chapter_hit(chapter["url"])
-        except Exception as e:
-            print(f"[SKIP] {chapter['name']}: {e}")
-            continue
+def cache_fixed_hit(
+    url: str,
+    chapter_name: str,
+    novel_url: str,
+    global_lore: dict,
+    refresh_mode: str = "none",
+) -> str:
+    if refresh_mode == "none":
+        row = db_fetch_one("SELECT fixed_html_text FROM chapter_page WHERE url = ?", (url,))
+        if row is not None and row[0] is not None:
+            tqdm.write(f"Fixed HTML cache hit: {url}")
+            return str(row[0])
+
+    # Retrieve clean html first
+    clean_html = cache_clean_hit(url, refresh_mode="none" if refresh_mode != "all" else "all")
+    fixed_html = fix_chapter(
+        raw_cleaned_html=clean_html,
+        chapter_name=chapter_name,
+        novel_url=novel_url,
+        global_lore=global_lore,
+    )
+
+    db_execute(
+        "UPDATE chapter_page SET fixed_html_text = ? WHERE url = ?",
+        (fixed_html, url),
+    )
+
+    return fixed_html
 
 
-def clean_chapters(metadata: dict) -> None:
-    for chapter in iter_chapters(metadata):
-        print(f"[{chapter['volume']}] Cleaning: {chapter['name']}")
+def scrape_chapters(
+    metadata: dict,
+    refresh_mode: str = "none",
+    progress_callback=None,
+    cancel_event=None,
+) -> None:
+    chapters = list(iter_chapters(metadata))
+    total = len(chapters)
+    if progress_callback is None:
+        iterator = tqdm(chapters, desc="Scraping chapters", unit="ch")
+    else:
+        iterator = chapters
+    for idx, chapter in enumerate(iterator, start=1):
+        if cancel_event is not None and cancel_event.is_set():
+            raise InterruptedError("Scraping was cancelled by user")
         try:
-            cache_clean_hit(chapter["url"])
+            cache_chapter_hit(chapter["url"], refresh_mode=refresh_mode)
+            if progress_callback:
+                progress_callback(stage="scraping", current=idx, total=total, message=chapter.get("name") or chapter.get("url") or "")
+        except InterruptedError:
+            raise
         except Exception as e:
-            print(f"[SKIP] {chapter['name']}: {e}")
-            continue
+            tqdm.write(f"[SKIP] {chapter['name']}: {e}")
+            if progress_callback:
+                progress_callback(stage="scraping", current=idx, total=total, message=f"[SKIP] {chapter.get('name')}: {e}")
+
+
+def clean_chapters(
+    metadata: dict,
+    refresh_mode: str = "none",
+    progress_callback=None,
+    cancel_event=None,
+) -> None:
+    chapters = list(iter_chapters(metadata))
+    total = len(chapters)
+    if progress_callback is None:
+        iterator = tqdm(chapters, desc="Cleaning chapters", unit="ch")
+    else:
+        iterator = chapters
+    for idx, chapter in enumerate(iterator, start=1):
+        if cancel_event is not None and cancel_event.is_set():
+            raise InterruptedError("Cleaning was cancelled by user")
+        try:
+            cache_clean_hit(chapter["url"], refresh_mode=refresh_mode)
+            if progress_callback:
+                progress_callback(stage="cleaning", current=idx, total=total, message=chapter.get("name") or chapter.get("url") or "")
+        except InterruptedError:
+            raise
+        except Exception as e:
+            tqdm.write(f"[SKIP] {chapter['name']}: {e}")
+            if progress_callback:
+                progress_callback(stage="cleaning", current=idx, total=total, message=f"[SKIP] {chapter.get('name')}: {e}")
+
+
+def fix_chapters(
+    metadata: dict,
+    refresh_mode: str = "none",
+    progress_callback=None,
+    cancel_event=None,
+) -> None:
+    chapters = list(iter_chapters(metadata))
+    total = len(chapters)
+    novel_url = metadata.get("novel_url") or ""
+
+    # Sample the first chapter for initial lore extraction if needed
+    first_chapter_sample = ""
+    if chapters:
+        try:
+            first_raw = cache_chapter_hit(chapters[0]["url"], refresh_mode="none")
+            first_chapter_sample = clean_novel_html(first_raw, base_url=chapters[0]["url"])
+        except Exception:
+            pass
+
+    global_lore = get_or_init_novel_lore(novel_url, metadata, first_chapter_sample)
+
+    if progress_callback is None:
+        iterator = tqdm(chapters, desc="Fixing MTL chapters", unit="ch")
+    else:
+        iterator = chapters
+    for idx, chapter in enumerate(iterator, start=1):
+        if cancel_event is not None and cancel_event.is_set():
+            raise InterruptedError("Fixing was cancelled by user")
+        try:
+            cache_fixed_hit(
+                url=chapter["url"],
+                chapter_name=chapter.get("name") or f"Chapter {idx}",
+                novel_url=novel_url,
+                global_lore=global_lore,
+                refresh_mode=refresh_mode,
+            )
+            if progress_callback:
+                progress_callback(stage="fixing", current=idx, total=total, message=chapter.get("name") or chapter.get("url") or "")
+        except InterruptedError:
+            raise
+        except Exception as e:
+            tqdm.write(f"[SKIP] {chapter['name']}: {e}")
+            if progress_callback:
+                progress_callback(stage="fixing", current=idx, total=total, message=f"[SKIP] {chapter.get('name')}: {e}")

src/clean_html.py

@@ -13,7 +13,7 @@ def clean_novel_html(raw_html: str, base_url: str = "") -> str:
 
     soup = BeautifulSoup(raw_html, "lxml")
 
-    for comment in soup.find_all(text=lambda text: isinstance(text, Comment)):
+    for comment in soup.find_all(string=lambda text: isinstance(text, Comment)):
         comment.extract()
 
     invisible_tags = [
@@ -70,15 +70,17 @@ def clean_novel_html(raw_html: str, base_url: str = "") -> str:
                 tag.attrs = {}
         elif tag.name == "img":
             src = (
-                tag.get("src")
-                or tag.get("data-src")
+                tag.get("data-src")
+                or tag.get("data-orig-file")
                 or tag.get("data-original")
+                or tag.get("data-lazy-src")
+                or tag.get("src")
                 or ""
             ).strip()
             if src and not src.startswith("data:image"):
                 tag.attrs = {"src": urljoin(base_url, src) if base_url else src}
             else:
-                tag.attrs = {}
+                tag.decompose()
         else:
             tag.attrs = {}
 

src/database.py

@@ -1,30 +1,142 @@
+import os
 import sqlite3
+from contextlib import contextmanager
 from pathlib import Path
 
 SCRIPT_DIR = Path(__file__).resolve().parent.parent
 DB_PATH = str(SCRIPT_DIR / "sqlite.db")
+DATABASE_URL = os.environ.get("DATABASE_URL", "").strip()
+USE_PG = DATABASE_URL.startswith("postgres://") or DATABASE_URL.startswith("postgresql://")
+
+if USE_PG and DATABASE_URL.startswith("postgres://"):
+    DATABASE_URL = DATABASE_URL.replace("postgres://", "postgresql://", 1)
+
+
+def _adapt_query(query: str) -> str:
+    if USE_PG:
+        return query.replace("?", "%s")
+    return query
+
+
+@contextmanager
+def get_conn():
+    if USE_PG:
+        import psycopg
+
+        conn = psycopg.connect(DATABASE_URL)
+        try:
+            yield conn
+            conn.commit()
+        finally:
+            conn.close()
+    else:
+        conn = sqlite3.connect(DB_PATH)
+        try:
+            yield conn
+            conn.commit()
+        finally:
+            conn.close()
+
+
+def db_fetch_one(query: str, params: tuple = ()):
+    q = _adapt_query(query)
+    with get_conn() as conn:
+        cur = conn.cursor()
+        cur.execute(q, params)
+        return cur.fetchone()
+
+
+def db_fetch_all(query: str, params: tuple = ()):
+    q = _adapt_query(query)
+    with get_conn() as conn:
+        cur = conn.cursor()
+        cur.execute(q, params)
+        return cur.fetchall()
+
+
+def db_execute(query: str, params: tuple = ()):
+    q = _adapt_query(query)
+    with get_conn() as conn:
+        cur = conn.cursor()
+        cur.execute(q, params)
+
 
 def init_db() -> None:
-    with sqlite3.connect(DB_PATH) as conn:
-        conn.execute("PRAGMA journal_mode=WAL;")
-        conn.execute("PRAGMA synchronous=NORMAL;")
-        
-        conn.execute("""
-            CREATE TABLE IF NOT EXISTS main_page (
-                url TEXT PRIMARY KEY,
-                status_code INTEGER,
-                html_text TEXT,
-                json_text TEXT,
-                fetched_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-            )
-        """)
+    if USE_PG:
+        import psycopg
 
-        conn.execute("""
-            CREATE TABLE IF NOT EXISTS chapter_page (
-                url TEXT PRIMARY KEY,
-                status_code INTEGER,
-                html_text TEXT,
-                clean_html_text TEXT,
-                fetched_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-            )
-        """)
+        conn = psycopg.connect(DATABASE_URL)
+        try:
+            with conn.cursor() as cur:
+                cur.execute("""
+                    CREATE TABLE IF NOT EXISTS main_page (
+                        url TEXT PRIMARY KEY,
+                        status_code INTEGER,
+                        html_text TEXT,
+                        json_text TEXT,
+                        fetched_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
+                    )
+                """)
+                cur.execute("""
+                    CREATE TABLE IF NOT EXISTS chapter_page (
+                        url TEXT PRIMARY KEY,
+                        status_code INTEGER,
+                        html_text TEXT,
+                        clean_html_text TEXT,
+                        fixed_html_text TEXT,
+                        fetched_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
+                    )
+                """)
+                cur.execute("""
+                    CREATE TABLE IF NOT EXISTS novel_lore (
+                        novel_url TEXT PRIMARY KEY,
+                        lore_json TEXT,
+                        rolling_summary_json TEXT,
+                        updated_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
+                    )
+                """)
+                # Handle existing tables missing columns
+                try:
+                    cur.execute("ALTER TABLE chapter_page ADD COLUMN IF NOT EXISTS fixed_html_text TEXT;")
+                except Exception:
+                    pass
+            conn.commit()
+        finally:
+            conn.close()
+    else:
+        with sqlite3.connect(DB_PATH) as conn:
+            conn.execute("PRAGMA journal_mode=WAL;")
+            conn.execute("PRAGMA synchronous=NORMAL;")
+            conn.execute("""
+                CREATE TABLE IF NOT EXISTS main_page (
+                    url TEXT PRIMARY KEY,
+                    status_code INTEGER,
+                    html_text TEXT,
+                    json_text TEXT,
+                    fetched_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+                )
+            """)
+            conn.execute("""
+                CREATE TABLE IF NOT EXISTS chapter_page (
+                    url TEXT PRIMARY KEY,
+                    status_code INTEGER,
+                    html_text TEXT,
+                    clean_html_text TEXT,
+                    fixed_html_text TEXT,
+                    fetched_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+                )
+            """)
+            conn.execute("""
+                CREATE TABLE IF NOT EXISTS novel_lore (
+                    novel_url TEXT PRIMARY KEY,
+                    lore_json TEXT,
+                    rolling_summary_json TEXT,
+                    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+                )
+            """)
+            # Migration check for SQLite
+            cur = conn.cursor()
+            cur.execute("PRAGMA table_info(chapter_page);")
+            columns = [row[1] for row in cur.fetchall()]
+            if "fixed_html_text" not in columns:
+                conn.execute("ALTER TABLE chapter_page ADD COLUMN fixed_html_text TEXT;")

src/epub_creator.py

@@ -0,0 +1,314 @@
+import os
+import re
+from io import BytesIO
+
+from bs4 import BeautifulSoup
+from ebooklib import epub
+from PIL import Image
+from tqdm import tqdm
+
+from .database import db_fetch_one
+from .parse import robust_fetch_bytes
+
+
+_STYLE = """
+@namespace epub "http://www.idpf.org/2007/ops";
+body {
+    font-family: sans-serif;
+    font-size: 100%;
+    text-align: left;
+    padding-left: 5%;
+    padding-right: 5%;
+    line-height: 1.5;
+}
+p {
+    margin-top: 0;
+    margin-bottom: 1em;
+    text-indent: 1.5em;
+}
+h1 {
+    text-align: center;
+    text-transform: uppercase;
+    margin-bottom: 2em;
+}
+img {
+    max-width: 100%;
+    height: auto;
+    display: block;
+    margin: 1em auto;
+}
+hr {
+    border: 0;
+    border-top: 1px solid #ccc;
+    margin: 2em 0;
+}
+aside.tl-note, blockquote.tl-note {
+    font-size: 90%;
+    margin: 1.5em 1em;
+    padding: 0.6em 1.2em;
+    border-left: 3px solid #888;
+    background-color: #f5f5f5;
+    color: #444;
+}
+blockquote.tl-note p {
+    margin: 0;
+    text-indent: 0;
+}
+"""
+
+
+def _slugify(text: str) -> str:
+    slug = re.sub(r"[^\w\s-]", "", text.lower())
+    slug = re.sub(r"[-\s]+", "-", slug).strip("-_")
+    return slug or "novel"
+
+
+def _get_chapter_html(url: str, is_fixed: bool = False) -> str:
+    if is_fixed:
+        row = db_fetch_one("SELECT fixed_html_text, clean_html_text FROM chapter_page WHERE url = ?", (url,))
+        if row is not None:
+            if row[0]:
+                return str(row[0])
+            if row[1]:
+                return str(row[1])
+        raise RuntimeError(f"No fixed or cleaned HTML cached for {url}. Run fix_chapters first.")
+
+    row = db_fetch_one("SELECT clean_html_text FROM chapter_page WHERE url = ?", (url,))
+    if row is None or row[0] is None:
+        raise RuntimeError(
+            f"No cleaned HTML cached for {url}. Run clean_chapters first."
+        )
+    return str(row[0])
+
+
+def _download(url: str, referer: str | None = None) -> bytes:
+    status_code, content = robust_fetch_bytes(url, referer=referer)
+    if status_code != 200:
+        raise RuntimeError(f"Request to {url} failed with status code {status_code}")
+    return content
+
+
+_MAX_IMAGE_WIDTH = int(os.environ.get("IMAGE_MAX_WIDTH", "1600"))
+_MAX_IMAGE_HEIGHT = int(os.environ.get("IMAGE_MAX_HEIGHT", "2400"))
+_JPEG_QUALITY = int(os.environ.get("IMAGE_JPEG_QUALITY", "85"))
+
+
+def _has_transparency(image: Image.Image) -> bool:
+    if image.mode in ("RGBA", "LA"):
+        alpha = image.getchannel("A") if "A" in image.getbands() else None
+        if alpha is not None:
+            min_alpha, _ = alpha.getextrema()
+            return min_alpha < 255
+    if image.mode == "P" and "transparency" in image.info:
+        return True
+    return False
+
+
+def _compress_image(data: bytes) -> tuple[bytes, str]:
+    try:
+        with Image.open(BytesIO(data)) as image:
+            image.load()
+            fmt = (image.format or "").upper()
+
+            if fmt == "GIF" and getattr(image, "is_animated", False):
+                return data, "gif"
+
+            was_resized = False
+            orig_w, orig_h = image.size
+            if orig_w > _MAX_IMAGE_WIDTH or orig_h > _MAX_IMAGE_HEIGHT:
+                image.thumbnail(
+                    (_MAX_IMAGE_WIDTH, _MAX_IMAGE_HEIGHT),
+                    Image.Resampling.LANCZOS,
+                )
+                was_resized = True
+
+            buffer = BytesIO()
+            has_alpha = _has_transparency(image)
+
+            if has_alpha:
+                image.save(buffer, format="PNG", optimize=True)
+                compressed = buffer.getvalue()
+                if was_resized or len(compressed) < len(data):
+                    return compressed, "png"
+                return data, "png"
+
+            if image.mode != "RGB":
+                image = image.convert("RGB")
+            image.save(
+                buffer,
+                format="JPEG",
+                quality=_JPEG_QUALITY,
+                optimize=True,
+            )
+            compressed = buffer.getvalue()
+
+            if fmt in ("JPEG", "JPG"):
+                if was_resized or len(compressed) < len(data):
+                    return compressed, "jpg"
+                return data, "jpg"
+
+            return compressed, "jpg"
+    except Exception:
+        return data, "jpg"
+
+
+def _embed_images(
+    book: epub.EpubBook, html: str, image_cache: dict, referer: str | None = None
+) -> str:
+    soup = BeautifulSoup(html, "lxml")
+
+    for img_tag in soup.find_all("img"):
+        src = img_tag.get("src")
+        if not src:
+            continue
+
+        if src in image_cache:
+            img_tag["src"] = image_cache[src]
+            continue
+
+        try:
+            raw = _download(src, referer=referer)
+            content, ext = _compress_image(raw)
+        except Exception as e:
+            tqdm.write(f"Error downloading/compressing image {src}: {e}")
+            continue
+
+        image_id = f"img_{len(image_cache) + 1}"
+        file_name = f"images/{image_id}.{ext}"
+
+        epub_img = epub.EpubImage()
+        epub_img.uid = image_id
+        epub_img.file_name = file_name
+        epub_img.content = content
+        book.add_item(epub_img)
+
+        img_tag["src"] = file_name
+        image_cache[src] = file_name
+
+    body = soup.body if soup.body is not None else soup
+    return body.decode_contents()
+
+
+def create_epub(
+    metadata: dict,
+    output_dir: str = "output",
+    progress_callback=None,
+    cancel_event=None,
+    is_fixed: bool = False,
+) -> str:
+    series = metadata.get("metadata", {})
+    title = series.get("title") or "Untitled"
+    if is_fixed and not title.endswith(" [Fixed]"):
+        book_title = f"{title} [Fixed]"
+    else:
+        book_title = title
+
+    author = series.get("author") or "Unknown"
+    language = series.get("language") or "en"
+    cover_url = series.get("cover_url")
+
+    book = epub.EpubBook()
+    book.set_identifier(_slugify(book_title))
+    book.set_title(book_title)
+    book.set_language(language)
+    book.add_author(author)
+
+    if cover_url:
+        try:
+            cover_raw = _download(cover_url, referer=metadata.get("novel_url"))
+            cover_bytes, cover_ext = _compress_image(cover_raw)
+            book.set_cover(f"cover.{cover_ext}", cover_bytes)
+        except Exception as e:
+            print(f"Cover download failed ({cover_url}): {e}. Proceeding without it.")
+
+    nav_css = epub.EpubItem(
+        uid="style_nav",
+        file_name="style/nav.css",
+        media_type="text/css",
+        content=_STYLE,
+    )
+    book.add_item(nav_css)
+
+    image_cache = {}
+    toc_structure = []
+    spine_structure = ["nav"]
+    chapter_counter = 1
+
+    volumes = metadata.get("volumes", [])
+    total_chapters = sum(len(v.get("chapters", [])) for v in volumes)
+
+    def _process_volume(volume, pbar=None):
+        nonlocal chapter_counter
+        volume_name = volume.get("name", "Volume")
+        sections = []
+
+        for chapter in volume.get("chapters", []):
+            if cancel_event is not None and cancel_event.is_set():
+                raise InterruptedError("EPUB compilation was cancelled by user")
+
+            chapter_name = chapter.get("name") or "Untitled"
+            url = chapter.get("url")
+
+            if pbar is not None:
+                pbar.set_postfix_str(chapter_name[:30])
+                pbar.update(1)
+
+            if progress_callback is not None:
+                progress_callback(stage="epub", current=chapter_counter, total=total_chapters, message=chapter_name)
+
+            if not url:
+                tqdm.write(f"[SKIP] No URL for chapter: {chapter_name}")
+                chapter_counter += 1
+                continue
+
+            try:
+                chapter_html = _get_chapter_html(url, is_fixed=is_fixed)
+            except RuntimeError as e:
+                tqdm.write(f"[SKIP] {chapter_name}: {e}")
+                chapter_counter += 1
+                continue
+
+            chapter_html = _embed_images(book, chapter_html, image_cache, referer=url)
+
+            file_name = f"chapter_{chapter_counter}.xhtml"
+            epub_chapter = epub.EpubHtml(
+                title=chapter_name,
+                file_name=file_name,
+                lang=language,
+            )
+            epub_chapter.add_item(nav_css)
+            epub_chapter.content = f"<h1>{chapter_name}</h1>\n{chapter_html}"
+
+            book.add_item(epub_chapter)
+            sections.append(epub_chapter)
+            spine_structure.append(epub_chapter)
+
+            chapter_counter += 1
+
+        if sections:
+            toc_structure.append((epub.Section(volume_name), tuple(sections)))
+
+    if progress_callback is None:
+        with tqdm(total=total_chapters, desc="Building EPUB", unit="ch") as pbar:
+            for volume in volumes:
+                _process_volume(volume, pbar=pbar)
+    else:
+        for volume in volumes:
+            _process_volume(volume, pbar=None)
+
+    if not spine_structure[1:]:
+        raise RuntimeError("No chapters were added to the EPUB.")
+
+    book.toc = toc_structure
+    book.add_item(epub.EpubNav())
+    book.add_item(epub.EpubNcx())
+    book.spine = spine_structure
+
+    os.makedirs(output_dir, exist_ok=True)
+    slug = _slugify(title)
+    filename = f"{slug}-fixed.epub" if is_fixed else f"{slug}.epub"
+    output_path = os.path.join(output_dir, filename)
+
+    epub.write_epub(output_path, book, {})
+    print(f"Successfully generated: {output_path}")
+    return output_path

src/llm.py

@@ -0,0 +1,51 @@
+import os
+from openai import OpenAI
+from dotenv import load_dotenv
+
+
+load_dotenv()
+
+MODEL = os.environ.get("LLM_MODEL")
+if not MODEL:
+    raise RuntimeError("LLM_MODEL is not set. Add it to your .env file.")
+
+BASE_URL = os.environ.get("LLM_BASE_URL")
+if not BASE_URL:
+    raise RuntimeError("LLM_BASE_URL is not set. Add it to your .env file.")
+
+MAX_TOKENS = int(os.environ.get("LLM_MAX_TOKENS", "65536"))
+
+API_KEY = os.environ.get("LLM_API_KEY")
+if not API_KEY:
+    raise RuntimeError("LLM_API_KEY is not set. Add it to your .env file.")
+
+client = OpenAI(
+    api_key=API_KEY,
+    base_url=BASE_URL,
+)
+
+
+def _request(system_prompt: str, user_content: str, response_format=None) -> str:
+    kwargs = {
+        "model": MODEL,
+        "messages": [
+            {"role": "system", "content": system_prompt},
+            {"role": "user", "content": user_content},
+        ],
+        "temperature": 0.0,
+        "max_tokens": MAX_TOKENS,
+        "extra_body": {"thinking": {"type": "disabled"}},
+    }
+    if response_format is not None:
+        kwargs["response_format"] = response_format
+
+    response = client.chat.completions.create(**kwargs)
+    return (response.choices[0].message.content or "").strip()
+
+
+def generate_json(system_prompt: str, user_content: str) -> str:
+    return _request(system_prompt, user_content, response_format={"type": "json_object"})
+
+
+def generate_text(system_prompt: str, user_content: str) -> str:
+    return _request(system_prompt, user_content)

src/metadata.py

@@ -1,60 +1,49 @@
 import json
-import os
-import re
-import sqlite3
-from google import genai
-from google.genai import types
-from dotenv import load_dotenv
-from .database import DB_PATH
+from .database import db_execute, db_fetch_one
 from .parse import cache_hit
 from .clean_html import clean_novel_html
 from .prompt import INDEX_EXTRACTION_SYSTEM_PROMPT
+from .llm import generate_json
 
 
-load_dotenv()
-client = genai.Client()
+def extract_metadata_json(
+    html_content: str, url: str, refresh_mode: str = "none"
+) -> dict:
+    if isinstance(refresh_mode, bool):
+        refresh_mode = "all" if refresh_mode else "none"
 
-
-def extract_metadata_json(html_content: str, url: str) -> dict:
-    with sqlite3.connect(DB_PATH) as conn:
-        cursor = conn.cursor()
-        cursor.execute("SELECT json_text FROM main_page WHERE url = ?", (url,))
-        row = cursor.fetchone()
+    if refresh_mode == "none":
+        row = db_fetch_one("SELECT json_text FROM main_page WHERE url = ?", (url,))
         if row is not None and row[0] is not None:
-            print("Metadata JSON Cache hit!")
-            return json.loads(row[0])
+            data = json.loads(row[0])
+            data["novel_url"] = url
+            return data
 
-    response = client.models.generate_content(
-        model="gemini-3.6-flash",
-        contents=html_content,
-        config=types.GenerateContentConfig(
-            system_instruction=INDEX_EXTRACTION_SYSTEM_PROMPT,
-            temperature=0.0,
-            response_mime_type="application/json",
-            tools=None
-        ),
-    )
-    raw_output = response.text.strip()
+    raw_output = generate_json(INDEX_EXTRACTION_SYSTEM_PROMPT, html_content)
 
-    with sqlite3.connect(DB_PATH) as conn:
-        conn.execute("""
+    db_execute("""
             INSERT INTO main_page (url, json_text)
             VALUES (?, ?)
             ON CONFLICT(url) DO UPDATE SET json_text = excluded.json_text
         """, (url, raw_output))
-        conn.commit()
 
-    return json.loads(raw_output)
+    data = json.loads(raw_output)
+    data["novel_url"] = url
+    return data
 
 
-def write_json(page_url: str, output_json_path: str) -> None:
-    html_content = clean_novel_html(cache_hit(page_url))
+def write_json(
+    page_url: str, output_json_path: str, refresh_mode: str = "none"
+) -> None:
+    html_content = clean_novel_html(cache_hit(page_url, refresh_mode=refresh_mode), base_url=page_url)
     print("Extracting metadata via LLM...")
 
-    extracted_data = extract_metadata_json(html_content, page_url)
+    extracted_data = extract_metadata_json(
+        html_content, page_url, refresh_mode=refresh_mode
+    )
     with open(output_json_path, "w", encoding="utf-8") as f:
         json.dump(extracted_data, f, indent=2, ensure_ascii=False)
     print(f"\n[SUCCESS] Extracted JSON saved to: {output_json_path}\n")
 
 if __name__ == "__main__":
-    pass
+    pass

src/parse.py

@@ -1,17 +1,22 @@
 import random
+import re
 import time
-import sqlite3
+from urllib.parse import urljoin
 from curl_cffi import requests
 from curl_cffi.requests.errors import RequestsError
-from .database import DB_PATH
-from pathlib import Path
+from .database import db_execute, db_fetch_one
 
 
 session = requests.Session(impersonate="chrome120")
 
-def robust_fetch(
-    url: str, min_delay: float = 2.0, max_delay: float = 5.0, retries: int = 3
-) -> str:
+
+def _fetch_response(
+    url: str,
+    min_delay: float,
+    max_delay: float,
+    retries: int,
+    headers: dict | None = None,
+):
     sleep_time = random.uniform(min_delay, max_delay)
     if random.random() < 0.08:
         sleep_time += random.uniform(4.0, 8.0)
@@ -19,7 +24,7 @@ def robust_fetch(
 
     for attempt in range(retries):
         try:
-            response = session.get(url, timeout=15)
+            response = session.get(url, headers=headers, timeout=15)
 
             if response.status_code in (429, 503):
                 retry_after = response.headers.get("Retry-After")
@@ -36,7 +41,7 @@ def robust_fetch(
                     "Cloudflare Managed Challenge encountered. Headless browser required."
                 )
 
-            return response.status_code, str(response.text)
+            return response
 
         except (RequestsError, TimeoutError) as e:
             if attempt == retries - 1:
@@ -48,25 +53,96 @@ def robust_fetch(
     )
 
 
-def cache_hit(url: str) -> str:
-    with sqlite3.connect(DB_PATH) as conn:
-        cursor = conn.cursor()
-        cursor.execute("SELECT html_text FROM main_page WHERE url = ?", (url,))
-        row = cursor.fetchone()
-        if row is not None:
-            print("Request cache hit!")
+def robust_fetch(
+    url: str,
+    min_delay: float = 2.0,
+    max_delay: float = 5.0,
+    retries: int = 3,
+    headers: dict | None = None,
+) -> tuple[int, str]:
+    response = _fetch_response(url, min_delay, max_delay, retries, headers=headers)
+    return response.status_code, str(response.text)
+
+
+def robust_fetch_bytes(
+    url: str,
+    min_delay: float = 2.0,
+    max_delay: float = 5.0,
+    retries: int = 3,
+    headers: dict | None = None,
+    referer: str | None = None,
+) -> tuple[int, bytes]:
+    req_headers = {
+        "Accept": "image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8",
+        "Sec-Fetch-Dest": "image",
+        "Sec-Fetch-Mode": "no-cors",
+        "Sec-Fetch-Site": "cross-site",
+    }
+    if referer:
+        req_headers["Referer"] = referer
+    if headers:
+        req_headers.update(headers)
+
+    response = _fetch_response(
+        url, min_delay, max_delay, retries, headers=req_headers
+    )
+    return response.status_code, response.content
+
+# Exclusive to ZTL
+def _fetch_wp_manga_chapters(base_url: str, html: str) -> str:
+    manga_id_match = (
+        re.search(r'["\']manga_id["\']\s*:\s*["\']?(\d+)', html)
+        or re.search(r'["\']wc_post_id["\']\s*:\s*["\']?(\d+)', html)
+        or re.search(r'postid-(\d+)', html)
+    )
+    if not manga_id_match:
+        return ""
+
+    manga_id = manga_id_match.group(1)
+
+    ajax_url_match = re.search(r'["\']ajax_url["\']\s*:\s*["\'](http[^"\']+)', html)
+    if ajax_url_match:
+        ajax_url = ajax_url_match.group(1).replace(r"\/", "/")
+    else:
+        ajax_url = urljoin(base_url, "/wp-admin/admin-ajax.php")
+
+    try:
+        res = session.post(
+            ajax_url,
+            data={"action": "manga_get_chapters", "manga": manga_id},
+            timeout=15,
+        )
+        if res.status_code == 200:
+            return res.text
+    except Exception:
+        pass
+
+    return ""
+
+
+def cache_hit(url: str, refresh_mode: str = "none") -> str:
+    if isinstance(refresh_mode, bool):
+        refresh_mode = "all" if refresh_mode else "none"
+
+    if refresh_mode != "all":
+        row = db_fetch_one("SELECT html_text FROM main_page WHERE url = ?", (url,))
+        if row is not None and row[0] is not None:
             return str(row[0])
 
-    # Cache miss: fetch and save
     status_code, html = robust_fetch(url)
     if status_code != 200:
         raise RuntimeError(f"Request to {url} failed with status code {status_code}")
-    
-    with sqlite3.connect(DB_PATH) as conn:
-        conn.execute(
-            "INSERT OR REPLACE INTO main_page (url, status_code, html_text) VALUES (?, ?, ?)",
-            (url, status_code, html)
-        )
-        conn.commit()
 
-    return html
+    extra_chapters = _fetch_wp_manga_chapters(url, html)
+    if extra_chapters:
+        if "</body>" in html:
+            html = html.replace("</body>", f"<div class='injected-wp-manga-chapters'>{extra_chapters}</div></body>")
+        else:
+            html = f"{html}\n<div class='injected-wp-manga-chapters'>{extra_chapters}</div>"
+
+    db_execute(
+        "INSERT INTO main_page (url, status_code, html_text) VALUES (?, ?, ?) ON CONFLICT(url) DO UPDATE SET status_code=excluded.status_code, html_text=excluded.html_text, fetched_at=CURRENT_TIMESTAMP",
+        (url, status_code, html),
+    )
+
+    return html

src/prompt.py

@@ -53,7 +53,7 @@ TAG & FORMATTING RULES:
 
 TRANSLATOR & CONTEXT NOTES:
 1. KEEP informative translator notes that explain cultural context, puns, lore, or honorifics. Format them cleanly as:
-   <aside class="tl-note"><strong>Note:</strong> [Context explanation here]</aside>
+   <blockquote class="tl-note"><p><strong>TL Note:</strong> [Context explanation here]</p></blockquote>
 2. REMOVE translator banter, jokes, meme bloopers, editor rants, or off-topic conversational commentary.
 
 STRICT REMOVAL RULES:
@@ -64,5 +64,84 @@ STRICT REMOVAL RULES:
 STRICT FIDELITY RULES:
 1. DO NOT summarize, paraphrase, rewrite, or correct the author's prose or character dialogue.
 2. DO NOT delete legitimate story paragraphs.
-3. Output ONLY the sanitized HTML body content (<p>, <hr />, <img>, <aside>). Do not include <html>, <head>, <body> wrappers, or markdown code fences (```html).
-"""
+3. Output ONLY the sanitized HTML body content (<p>, <hr />, <img>, <blockquote>). Do not include <html>, <head>, <body> wrappers, or markdown code fences (```html).
+"""
+
+
+LORE_EXTRACTION_SYSTEM_PROMPT = """
+You are an expert light novel entity extraction and lore analysis engine.
+
+YOUR TASK:
+Analyze the provided novel description and initial chapters to extract key characters, proper terminology, and narrative rules into a standardized JSON glossary.
+
+EXTRACTION SCHEMA:
+{
+  "characters": [
+    {
+      "raw_name": "Original Japanese or Romanized Name",
+      "mtl_variants": ["Variant1", "Variant2"],
+      "standard_name": "Standardized English Name",
+      "gender": "male | female | unknown",
+      "role": "Protagonist | Ally | Antagonist | Supporting",
+      "speech_style": "Brief description of dialogue tone"
+    }
+  ],
+  "terms": {
+    "Original Term or MTL Term": "Standardized English Term"
+  },
+  "rules": {
+    "keep_honorifics": false,
+    "pov": "First-person | Third-person limited | Third-person omniscient"
+  }
+}
+
+RULES:
+1. Identify true recurring character names vs generic nouns.
+2. Standardize spelling variants (e.g., "Arus", "Arce", "Ars" -> "Ars").
+3. Identify faction names, skill/magic names, and location names.
+4. Output ONLY valid raw JSON.
+"""
+
+
+APE_SYSTEM_PROMPT = """
+You are an expert Japanese-to-English literary translator and copyeditor specializing in light novels and web novels. Your goal is to transform raw Machine Translation (MTL) into fluent, natural, and immersive English narrative prose while maintaining 100% factual accuracy to the source material.
+
+CORE OPERATIONAL RULES:
+1. STRICT CONTEXT ALIGNMENT (Zero Hallucination):
+   - Do NOT invent, add, or alter plot points, actions, dialogue beats, or world-building elements.
+   - Do NOT insert extra prose, backstory, or commentary that was not present or implied in the raw MTL text.
+   - If a sentence is completely incomprehensible, reconstruct the most logical meaning using surrounding paragraph context and the provided Glossary/Lore sheet.
+
+2. FIDELITY-PRESERVING STYLISTIC POLISHING:
+   - Eliminate common MTL artifacts: awkward literal idioms, stiff sentence structures, repetitive word choices, and robotic cadence.
+   - Fix dropped or inverted pronouns (Japanese subject-dropping often causes MTL to confuse "he/she/they/it"). Cross-reference active characters in the scene to ensure correct pronoun usage.
+   - Smooth out dialogue to sound natural when spoken aloud, matching character voice and tone as defined in the Glossary.
+   - Preserve narrative perspective (1st-person POV vs. 3rd-person limited) consistently throughout the scene.
+
+3. GLOSSARY & ENTITY ENFORCEMENT:
+   - Always strictly honor the provided Active Glossary & Character Sheet.
+   - Do not translate proper names, places, or technique names literally if an official/standardized English equivalent is specified in the Glossary.
+
+4. TRANSLATOR NOTES:
+   - Format any informative translator/editor notes strictly as:
+     <blockquote class="tl-note"><p><strong>TL Note:</strong> [Context explanation]</p></blockquote>
+   - Strip out off-topic translator jokes, memes, or donation links.
+
+5. OUTPUT SPECIFICATION:
+   - Output ONLY the polished chapter prose wrapped in standard <p>...</p> tags, with <hr /> for scene breaks and intact <img> tags.
+   - Do NOT duplicate the chapter title at the beginning.
+   - Do NOT include markdown code block wrappers (```html) or introductory conversational text.
+"""
+
+
+CHAPTER_SUMMARY_SYSTEM_PROMPT = """
+You are an expert story summarization engine.
+
+YOUR TASK:
+Read the provided chapter prose and generate an extremely concise 2-sentence summary of the main plot points, current location, active characters, and ending state.
+
+CONSTRAINTS:
+- Maximum 150 words / 200 tokens.
+- Factual and direct. Focus on character actions and scene transitions.
+- Output ONLY the summary text, with no introductory text or labels.
+"""

tests/test_frontend.py

@@ -0,0 +1,126 @@
+import pytest
+from fastapi.testclient import TestClient
+from web.app import app
+
+client = TestClient(app)
+
+def test_index_page_returns_200():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    assert "<!DOCTYPE html>" in html
+    assert "/static/app.css" in html
+    assert "Silksoup: Novel to EPUB" in html
+
+def test_dark_mode_only():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    assert '<html lang="en" class="dark">' in html
+    assert "themeBtn" not in html
+    assert "toggleTheme" not in html
+
+def test_no_external_cdn_dependencies():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    assert "cdn.tailwindcss.com" not in html
+    assert "googleapis" not in html
+    assert "cloudflare" not in html
+
+def test_no_em_dash_in_template():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    # Rule R-02 ban
+    assert "—" not in html
+
+def test_static_app_css_available_and_contains_azure_tokens():
+    response = client.get("/static/app.css")
+    assert response.status_code == 200
+    css = response.text
+    assert "--bg: #141517" in css
+    assert "--surface: #1C1E21" in css
+    assert "--primary: #0284C7" in css
+    assert "--primary-accent: #38BDF8" in css
+    assert ":focus-visible" in css
+    assert "ui-monospace" in css
+
+def test_required_dom_contract_elements_exist():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    required_ids = [
+        "jobForm",
+        "urlInput",
+        "startBtn",
+        "btnSpinner",
+        "btnText",
+        "fixCheck",
+        "activeJob",
+        "cancelJobBtn",
+        "statusPulse",
+        "jobStatusBadge",
+        "jobStage",
+        "jobMessage",
+        "jobCounter",
+        "jobProgress",
+        "jobLog",
+        "autoScrollCheck",
+        "expandLogBtn",
+        "jobResult",
+        "downloadLink",
+        "searchEpub",
+        "epubCount",
+        "epubList",
+        "epubEmpty",
+        "toastContainer",
+        "notifyBtn",
+    ]
+    for element_id in required_ids:
+        assert f'id="{element_id}"' in html, f"Missing required element id: {element_id}"
+
+def test_aria_accessibility_attributes_present():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    assert 'role="progressbar"' in html
+    assert 'aria-valuenow' in html
+    assert 'aria-live="polite"' in html
+    assert 'role="radiogroup"' in html
+
+def test_form_submits_on_submit_event():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    assert "submitJob()" in html
+    assert "addEventListener('submit'" in html
+    assert 'type="submit"' in html
+
+def test_guide_details_present():
+    response = client.get("/")
+    assert response.status_code == 200
+    html = response.text
+    assert "<details" in html
+    assert "Workflow Pipeline Guide" in html
+    assert "1. Ingest & Cache" in html
+    assert "4. Build EPUB" in html
+
+def test_api_epubs_endpoint_unmodified():
+    response = client.get("/api/epubs")
+    assert response.status_code == 200
+    data = response.json()
+    assert "epubs" in data
+    assert "output_dir" in data
+
+def test_api_jobs_validation_unmodified():
+    response = client.post("/api/jobs", json={"url": "not-a-valid-url"})
+    assert response.status_code == 400
+
+def test_api_cancel_missing_job_returns_404():
+    response = client.post("/api/jobs/nonexistent-id/cancel")
+    assert response.status_code == 404
+
+def test_api_download_missing_epub_returns_404():
+    response = client.get("/api/epubs/nonexistent-book.epub")
+    assert response.status_code == 404

tests/test_singleton_pipeline.py

@@ -0,0 +1,130 @@
+import asyncio
+import pytest
+from fastapi.testclient import TestClient
+from web.app import app
+import web.job_manager as jm
+
+client = TestClient(app)
+
+
+@pytest.fixture(autouse=True)
+def reset_job_manager():
+    jm.cancel_all_jobs()
+    jm.jobs.clear()
+    jm.subscribers.clear()
+    jm.cancel_events.clear()
+    jm.active_job_id = None
+    yield
+    jm.cancel_all_jobs()
+    jm.jobs.clear()
+    jm.subscribers.clear()
+    jm.cancel_events.clear()
+    jm.active_job_id = None
+
+
+def test_active_job_endpoint_returns_idle_when_empty():
+    res = client.get("/api/jobs/active")
+    assert res.status_code == 200
+    data = res.json()
+    assert data["active"] is False
+    assert data["job"] is None
+
+
+def test_active_job_endpoint_returns_running_job():
+    job_id = jm.create_job("https://example.com/novel-1")
+    jm.jobs[job_id]["status"] = "running"
+    jm.jobs[job_id]["stage"] = "scraping"
+    jm.jobs[job_id]["current"] = 5
+    jm.jobs[job_id]["total"] = 20
+    jm.jobs[job_id]["message"] = "Scraping chapter 5"
+
+    res = client.get("/api/jobs/active")
+    assert res.status_code == 200
+    data = res.json()
+    assert data["active"] is True
+    assert data["job"]["id"] == job_id
+    assert data["job"]["stage"] == "scraping"
+    assert data["job"]["current"] == 5
+    assert data["job"]["total"] == 20
+    assert data["job"]["percentage"] == 25
+    assert data["job"]["message"] == "Scraping chapter 5"
+
+
+def test_concurrent_job_submission_returns_409():
+    job_id = jm.create_job("https://example.com/active-novel")
+    jm.jobs[job_id]["status"] = "running"
+
+    res = client.post("/api/jobs", json={"url": "https://example.com/another-novel"})
+    assert res.status_code == 409
+    assert "Another novel conversion is currently in progress" in res.json()["detail"]
+
+
+def test_fix_job_submission_returns_409_when_busy(tmp_path, monkeypatch):
+    test_epub = tmp_path / "test.epub"
+    test_epub.write_text("dummy epub")
+    monkeypatch.setattr("web.app.OUTPUT_DIR", tmp_path)
+    monkeypatch.setattr("web.app._find_novel_url_for_epub", lambda f: "https://example.com/test-novel")
+
+    job_id = jm.create_job("https://example.com/active-novel")
+    jm.jobs[job_id]["status"] = "running"
+
+    res = client.post("/api/epubs/test.epub/fix")
+    assert res.status_code == 409
+    assert "Another novel conversion is currently in progress" in res.json()["detail"]
+
+
+def test_cancellation_frees_worker_lock():
+    job_id = jm.create_job("https://example.com/active-novel")
+    jm.jobs[job_id]["status"] = "running"
+
+    assert jm.is_worker_busy() is True
+
+    cancel_res = client.post(f"/api/jobs/{job_id}/cancel")
+    assert cancel_res.status_code == 200
+    assert cancel_res.json()["status"] == "cancelled"
+
+    assert jm.is_worker_busy() is False
+
+    active_res = client.get("/api/jobs/active")
+    assert active_res.status_code == 200
+    assert active_res.json()["active"] is False
+
+    # A subsequent job can now be submitted
+    new_job_id = jm.create_job("https://example.com/subsequent-novel")
+    assert new_job_id is not None
+    assert jm.active_job_id == new_job_id
+
+
+def test_sse_pubsub_fanout_multiple_subscribers():
+    job_id = jm.create_job("https://example.com/sse-test")
+
+    q1 = jm.subscribe_job_events(job_id)
+    q2 = jm.subscribe_job_events(job_id)
+
+    assert len(jm.subscribers[job_id]) == 2
+
+    test_event = {"stage": "scraping", "current": 1, "total": 10, "message": "Chapter 1"}
+    jm._push(job_id, test_event)
+
+    assert not q1.empty()
+    assert not q2.empty()
+    assert q1.get_nowait() == test_event
+    assert q2.get_nowait() == test_event
+
+    jm.unsubscribe_job_events(job_id, q1)
+    assert len(jm.subscribers[job_id]) == 1
+
+    jm.unsubscribe_job_events(job_id, q2)
+    assert job_id not in jm.subscribers
+
+
+def test_frontend_has_sync_active_job_and_reconnection():
+    res = client.get("/")
+    assert res.status_code == 200
+    html = res.text
+
+    assert "syncActiveJob()" in html
+    assert "/api/jobs/active" in html
+    assert "setFormLocked(true)" in html
+    assert "setFormLocked(false)" in html
+    assert "409" in html

web/app.py

@@ -0,0 +1,295 @@
+import asyncio
+import json
+import os
+import re
+from contextlib import asynccontextmanager
+from pathlib import Path
+
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.responses import HTMLResponse, FileResponse, StreamingResponse
+from fastapi.staticfiles import StaticFiles
+from fastapi.templating import Jinja2Templates
+
+from src.database import db_fetch_one, db_fetch_all, init_db
+from src.epub_creator import _slugify
+from .job_manager import (
+    jobs,
+    create_job,
+    cancel_job,
+    cancel_all_jobs,
+    run_job,
+    get_active_job,
+    subscribe_job_events,
+    unsubscribe_job_events,
+    WorkerBusyError,
+)
+
+OUTPUT_DIR = Path(os.environ.get("OUTPUT_DIR", "output")).resolve()
+OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    init_db()
+    yield
+    cancel_all_jobs()
+
+
+app = FastAPI(title="Silksoup", lifespan=lifespan)
+
+BASE_DIR = Path(__file__).resolve().parent
+templates = Jinja2Templates(directory=str(BASE_DIR / "templates"))
+
+if (BASE_DIR / "static").exists():
+    app.mount("/static", StaticFiles(directory=str(BASE_DIR / "static")), name="static")
+
+
+@app.get("/", response_class=HTMLResponse)
+async def index(request: Request):
+    return templates.TemplateResponse(request, "index.html")
+
+
+def _find_novel_url_for_epub(filename: str) -> str | None:
+    base_slug = _slugify(re.sub(r"(-fixed)?\.epub$", "", filename, flags=re.I))
+    rows = db_fetch_all("SELECT url, json_text FROM main_page WHERE json_text IS NOT NULL")
+    for row in rows:
+        url, json_str = row[0], row[1]
+        try:
+            data = json.loads(json_str)
+            title = data.get("metadata", {}).get("title") or ""
+            if _slugify(title) == base_slug:
+                return url
+        except Exception:
+            continue
+    return None
+
+
+@app.get("/api/jobs/active")
+async def get_active_job_endpoint():
+    active = get_active_job()
+    if active is None:
+        return {"active": False, "job": None}
+    return {"active": True, "job": active}
+
+
+@app.post("/api/jobs")
+async def start_job(payload: dict):
+    url = (payload.get("url") or "").strip()
+    if not url or not (url.startswith("http://") or url.startswith("https://")):
+        raise HTTPException(status_code=400, detail="Valid URL required")
+
+    active = get_active_job()
+    if active is not None:
+        raise HTTPException(
+            status_code=409,
+            detail=f"Another novel conversion is currently in progress: {active.get('url')}",
+        )
+
+    # Check if a fixed version of this novel already exists in the library
+    row = db_fetch_one("SELECT json_text FROM main_page WHERE url = ?", (url,))
+    if row is not None and row[0]:
+        try:
+            data = json.loads(row[0])
+            title = data.get("metadata", {}).get("title") or ""
+            slug = _slugify(title)
+            fixed_file = OUTPUT_DIR / f"{slug}-fixed.epub"
+            if fixed_file.exists():
+                raise HTTPException(
+                    status_code=409,
+                    detail="Novel already exists in library (Fixed version present).",
+                )
+        except HTTPException:
+            raise
+        except Exception:
+            pass
+
+    refresh_mode = payload.get("refresh_mode")
+    if refresh_mode is None:
+        force_refresh = bool(payload.get("force_refresh", False))
+        refresh_mode = "all" if force_refresh else "none"
+    elif refresh_mode not in ("none", "llm", "all", "fix"):
+        refresh_mode = "none"
+
+    fix = bool(payload.get("fix", False)) or (refresh_mode == "fix")
+
+    try:
+        job_id = create_job(url, refresh_mode=refresh_mode, fix=fix, output_dir=str(OUTPUT_DIR))
+    except WorkerBusyError as e:
+        raise HTTPException(
+            status_code=409,
+            detail=f"Another novel conversion is currently in progress: {e.active_job.get('url')}",
+        )
+    asyncio.create_task(run_job(job_id))
+    return {"job_id": job_id, "status": "queued"}
+
+
+@app.post("/api/epubs/{filename}/fix")
+async def fix_library_epub(filename: str):
+    safe = Path(filename).name
+    target = OUTPUT_DIR / safe
+    if not target.exists() or not target.is_file():
+        raise HTTPException(status_code=404, detail="EPUB file not found")
+
+    if "-fixed.epub" in safe.lower():
+        raise HTTPException(status_code=400, detail="This EPUB is already a fixed version.")
+
+    active = get_active_job()
+    if active is not None:
+        raise HTTPException(
+            status_code=409,
+            detail=f"Another novel conversion is currently in progress: {active.get('url')}",
+        )
+
+    novel_url = _find_novel_url_for_epub(safe)
+    if not novel_url:
+        raise HTTPException(
+            status_code=404,
+            detail="Source novel URL not found in cache. Re-scrape the novel to fix.",
+        )
+
+    try:
+        job_id = create_job(novel_url, refresh_mode="none", fix=True, output_dir=str(OUTPUT_DIR))
+    except WorkerBusyError as e:
+        raise HTTPException(
+            status_code=409,
+            detail=f"Another novel conversion is currently in progress: {e.active_job.get('url')}",
+        )
+    asyncio.create_task(run_job(job_id))
+    return {"job_id": job_id, "status": "queued", "novel_url": novel_url}
+
+
+@app.post("/api/jobs/{job_id}/cancel")
+async def stop_job(job_id: str):
+    job = jobs.get(job_id)
+    if not job:
+        raise HTTPException(status_code=404, detail="Job not found")
+    if job.get("status") not in ("running", "queued"):
+        return {"status": job.get("status"), "message": "Job is not running"}
+    cancel_job(job_id)
+    return {"status": "cancelled", "message": "Job cancelled successfully"}
+
+
+@app.get("/api/jobs")
+async def list_jobs():
+    items = []
+    for job_id, job in sorted(jobs.items(), key=lambda item: item[1].get("created_at", 0), reverse=True):
+        items.append({
+            "id": job["id"],
+            "url": job["url"],
+            "refresh_mode": job.get("refresh_mode", "none"),
+            "status": job["status"],
+            "stage": job["stage"],
+            "current": job["current"],
+            "total": job["total"],
+            "message": job["message"],
+            "created_at": job.get("created_at"),
+            "result_path": job.get("result_path"),
+            "error": job.get("error"),
+        })
+    return {"jobs": items[:20]}
+
+
+@app.get("/api/jobs/{job_id}")
+async def get_job(job_id: str):
+    job = jobs.get(job_id)
+    if not job:
+        raise HTTPException(status_code=404, detail="Job not found")
+    return {
+        "id": job["id"],
+        "url": job["url"],
+        "refresh_mode": job.get("refresh_mode", "none"),
+        "status": job["status"],
+        "stage": job["stage"],
+        "current": job["current"],
+        "total": job["total"],
+        "message": job["message"],
+        "error": job["error"],
+        "result_path": job["result_path"],
+        "logs": job["logs"][-200:],
+    }
+
+
+@app.get("/api/jobs/{job_id}/events")
+async def job_events(job_id: str):
+    job = jobs.get(job_id)
+    if not job:
+        raise HTTPException(status_code=404, detail="Job not found")
+
+    async def event_gen():
+        q = subscribe_job_events(job_id)
+        try:
+            events = list(job.get("events", []))
+            for evt in events:
+                yield f"data: {json.dumps(evt)}\n\n"
+                if evt.get("message") == "__DONE__":
+                    return
+
+            if job.get("status") in ("completed", "failed", "cancelled"):
+                yield f"data: {json.dumps({'stage': 'done', 'message': '__DONE__'})}\n\n"
+                return
+
+            while True:
+                try:
+                    evt = await asyncio.wait_for(q.get(), timeout=15.0)
+                    yield f"data: {json.dumps(evt)}\n\n"
+                    if evt.get("message") == "__DONE__":
+                        break
+                except asyncio.TimeoutError:
+                    yield ": keepalive\n\n"
+                    if job.get("status") in ("completed", "failed", "cancelled"):
+                        break
+        finally:
+            unsubscribe_job_events(job_id, q)
+
+    return StreamingResponse(event_gen(), media_type="text/event-stream", headers={
+        "Cache-Control": "no-cache",
+        "Connection": "keep-alive",
+        "X-Accel-Buffering": "no",
+    })
+
+
+@app.get("/api/epubs")
+async def list_epubs():
+    files = []
+    if OUTPUT_DIR.exists():
+        for p in sorted(OUTPUT_DIR.glob("*.epub"), key=lambda x: x.stat().st_mtime, reverse=True):
+            stat = p.stat()
+            name = p.name
+            is_fixed = "-fixed.epub" in name.lower() or " [fixed]" in name.lower()
+            base_key = re.sub(r"(-fixed)?\.epub$", "", name, flags=re.I).strip("-_")
+            files.append({
+                "name": name,
+                "base_key": base_key,
+                "is_fixed": is_fixed,
+                "size": stat.st_size,
+                "size_human": _human_size(stat.st_size),
+                "mtime": stat.st_mtime,
+            })
+    return {"epubs": files, "output_dir": str(OUTPUT_DIR)}
+
+
+@app.get("/api/epubs/{filename}")
+async def download_epub(filename: str):
+    safe = Path(filename).name
+    target = OUTPUT_DIR / safe
+    if not target.exists() or not target.is_file():
+        raise HTTPException(status_code=404, detail="File not found")
+    return FileResponse(str(target), media_type="application/epub+zip", filename=safe)
+
+
+@app.delete("/api/epubs/{filename}")
+async def delete_epub(filename: str):
+    safe = Path(filename).name
+    target = OUTPUT_DIR / safe
+    if not target.exists():
+        raise HTTPException(status_code=404, detail="File not found")
+    target.unlink()
+    return {"deleted": safe}
+
+
+def _human_size(n: int) -> str:
+    for unit in ["B", "KB", "MB", "GB"]:
+        if n < 1024:
+            return f"{n:.1f} {unit}" if unit != "B" else f"{n} B"
+        n /= 1024
+    return f"{n:.1f} TB"

web/job_manager.py

@@ -0,0 +1,252 @@
+import asyncio
+import threading
+import time
+import uuid
+from pathlib import Path
+
+from src.database import init_db
+from src.parse import cache_hit
+from src.clean_html import clean_novel_html
+from src.metadata import extract_metadata_json
+from src.chapter import clean_chapters, fix_chapters, scrape_chapters
+from src.epub_creator import create_epub
+
+jobs: dict[str, dict] = {}
+subscribers: dict[str, set[asyncio.Queue]] = {}
+cancel_events: dict[str, threading.Event] = {}
+active_job_id: str | None = None
+
+
+class WorkerBusyError(Exception):
+    def __init__(self, active_job: dict):
+        self.active_job = active_job
+        super().__init__("Worker is currently busy with another job")
+
+
+def get_active_job() -> dict | None:
+    if active_job_id is None:
+        return None
+    job = jobs.get(active_job_id)
+    if not job or job.get("status") not in ("queued", "running"):
+        return None
+    total = job.get("total") or 0
+    current = job.get("current") or 0
+    percentage = round(current / total * 100) if total else 0
+    return {
+        "id": job["id"],
+        "url": job["url"],
+        "refresh_mode": job.get("refresh_mode", "none"),
+        "fix": job.get("fix", False),
+        "status": job["status"],
+        "stage": job["stage"],
+        "current": current,
+        "total": total,
+        "percentage": percentage,
+        "message": job.get("message", ""),
+        "created_at": job.get("created_at"),
+        "result_path": job.get("result_path"),
+        "error": job.get("error"),
+        "logs": job.get("logs", [])[-200:],
+    }
+
+
+def is_worker_busy() -> bool:
+    return get_active_job() is not None
+
+
+def subscribe_job_events(job_id: str) -> asyncio.Queue:
+    q = asyncio.Queue(maxsize=1000)
+    if job_id not in subscribers:
+        subscribers[job_id] = set()
+    subscribers[job_id].add(q)
+    return q
+
+
+def unsubscribe_job_events(job_id: str, q: asyncio.Queue):
+    if job_id in subscribers:
+        subscribers[job_id].discard(q)
+        if not subscribers[job_id]:
+            subscribers.pop(job_id, None)
+
+
+def _push(job_id: str, event: dict):
+    job = jobs.get(job_id)
+    if job is None:
+        return
+    job["events"].append(event)
+    job["updated_at"] = time.time()
+    if "stage" in event:
+        job["stage"] = event["stage"]
+    if "current" in event:
+        job["current"] = event["current"]
+    if "total" in event:
+        job["total"] = event["total"]
+    if "message" in event:
+        job["message"] = event["message"]
+
+    queues = subscribers.get(job_id, set())
+    for q in list(queues):
+        try:
+            q.put_nowait(event)
+        except asyncio.QueueFull:
+            pass
+
+
+def create_job(url: str, refresh_mode: str = "none", fix: bool = False, output_dir: str = "output") -> str:
+    global active_job_id
+    active = get_active_job()
+    if active is not None:
+        raise WorkerBusyError(active)
+
+    job_id = uuid.uuid4().hex[:12]
+    active_job_id = job_id
+    jobs[job_id] = {
+        "id": job_id,
+        "url": url,
+        "refresh_mode": refresh_mode,
+        "fix": bool(fix),
+        "output_dir": output_dir,
+        "status": "queued",
+        "stage": "queued",
+        "current": 0,
+        "total": 0,
+        "message": "Queued",
+        "logs": [],
+        "events": [],
+        "result_path": None,
+        "error": None,
+        "created_at": time.time(),
+        "updated_at": time.time(),
+    }
+    subscribers[job_id] = set()
+    cancel_events[job_id] = threading.Event()
+    return job_id
+
+
+def cancel_job(job_id: str) -> bool:
+    global active_job_id
+    job = jobs.get(job_id)
+    if not job:
+        return False
+    evt = cancel_events.get(job_id)
+    if evt is not None:
+        evt.set()
+    job["status"] = "cancelled"
+    job["stage"] = "cancelled"
+    job["message"] = "Process stopped by user"
+    job["logs"].append("[CANCEL] Process stopped by user. Cached chapters remain safe.")
+    _push(job_id, {"stage": "cancelled", "current": job.get("current", 0), "total": job.get("total", 0), "message": "Process stopped by user"})
+    _push(job_id, {"stage": "done", "current": job.get("current", 0), "total": job.get("total", 0), "message": "__DONE__"})
+    if active_job_id == job_id:
+        active_job_id = None
+    return True
+
+
+def cancel_all_jobs():
+    global active_job_id
+    for job_id, evt in cancel_events.items():
+        job = jobs.get(job_id)
+        if job and job.get("status") in ("running", "queued"):
+            evt.set()
+            job["status"] = "cancelled"
+            job["stage"] = "cancelled"
+    active_job_id = None
+
+
+async def run_job(job_id: str):
+    global active_job_id
+    job = jobs.get(job_id)
+    if job is None:
+        return
+    url = job["url"]
+    refresh_mode = job.get("refresh_mode", "none")
+    is_fix_job = bool(job.get("fix") or refresh_mode == "fix")
+    output_dir = job["output_dir"]
+    cancel_evt = cancel_events.get(job_id)
+    job["status"] = "running"
+
+    def _cb(stage=None, current=None, total=None, message=None, **kwargs):
+        evt = {"stage": stage or "unknown", "current": current or 0, "total": total or 0, "message": message or ""}
+        _push(job_id, evt)
+        if message:
+            job["logs"].append(f"[{stage}] {message}")
+
+    try:
+        if cancel_evt and cancel_evt.is_set():
+            raise InterruptedError("Cancelled before start")
+
+        init_db()
+        _push(job_id, {"stage": "fetching", "current": 0, "total": 1, "message": "Fetching index page..."})
+        job["logs"].append(f"Fetching {url}")
+
+        html = await asyncio.to_thread(cache_hit, url, "none" if is_fix_job else refresh_mode)
+        if cancel_evt and cancel_evt.is_set():
+            raise InterruptedError("Cancelled after fetch")
+
+        cleaned = clean_novel_html(html, base_url=url)
+
+        _push(job_id, {"stage": "metadata", "current": 0, "total": 1, "message": "Extracting metadata via LLM..."})
+        job["logs"].append("Extracting metadata")
+
+        metadata = await asyncio.to_thread(extract_metadata_json, cleaned, url, "none" if is_fix_job else refresh_mode)
+        metadata["novel_url"] = url
+
+        if cancel_evt and cancel_evt.is_set():
+            raise InterruptedError("Cancelled after metadata")
+
+        volumes = metadata.get("volumes", [])
+        total_chapters = sum(len(v.get("chapters", [])) for v in volumes)
+
+        if is_fix_job:
+            _push(job_id, {"stage": "scraping", "current": 0, "total": total_chapters, "message": f"Ensuring {total_chapters} chapters are cached"})
+            await asyncio.to_thread(scrape_chapters, metadata, refresh_mode if refresh_mode != "fix" else "none", _cb, cancel_evt)
+
+            if cancel_evt and cancel_evt.is_set():
+                raise InterruptedError("Cancelled before APE fix")
+
+            _push(job_id, {"stage": "fixing", "current": 0, "total": total_chapters, "message": f"Fixing MTL for {total_chapters} chapters"})
+            job["logs"].append(f"Starting APE MTL repair on {total_chapters} chapters")
+            await asyncio.to_thread(fix_chapters, metadata, "none", _cb, cancel_evt)
+
+            if cancel_evt and cancel_evt.is_set():
+                raise InterruptedError("Cancelled before EPUB building")
+
+            _push(job_id, {"stage": "epub", "current": 0, "total": total_chapters, "message": "Building Fixed EPUB..."})
+            output_path = await asyncio.to_thread(create_epub, metadata, output_dir, _cb, cancel_evt, is_fixed=True)
+        else:
+            _push(job_id, {"stage": "cleaning", "current": 0, "total": total_chapters, "message": f"Cleaning {total_chapters} chapters"})
+            job["logs"].append(f"Found {len(volumes)} volumes, {total_chapters} chapters")
+
+            await asyncio.to_thread(clean_chapters, metadata, refresh_mode, _cb, cancel_evt)
+
+            if cancel_evt and cancel_evt.is_set():
+                raise InterruptedError("Cancelled before EPUB building")
+
+            _push(job_id, {"stage": "epub", "current": 0, "total": total_chapters, "message": "Building EPUB..."})
+            output_path = await asyncio.to_thread(create_epub, metadata, output_dir, _cb, cancel_evt, is_fixed=False)
+
+        job["result_path"] = output_path
+        job["status"] = "completed"
+        job["stage"] = "completed"
+        job["message"] = f"Completed: {Path(output_path).name}"
+        job["logs"].append(f"Generated {output_path}")
+        _push(job_id, {"stage": "completed", "current": total_chapters, "total": total_chapters, "message": job["message"]})
+        _push(job_id, {"stage": "done", "current": total_chapters, "total": total_chapters, "message": "__DONE__"})
+    except InterruptedError:
+        job["status"] = "cancelled"
+        job["stage"] = "cancelled"
+        job["message"] = "Process stopped by user"
+        job["logs"].append("[CANCEL] Process stopped by user. Cached chapters remain safe.")
+        _push(job_id, {"stage": "cancelled", "current": job.get("current", 0), "total": job.get("total", 0), "message": "Process stopped by user"})
+        _push(job_id, {"stage": "done", "current": job.get("current", 0), "total": job.get("total", 0), "message": "__DONE__"})
+    except Exception as e:
+        job["status"] = "failed"
+        job["stage"] = "failed"
+        job["error"] = str(e)
+        job["message"] = str(e)
+        job["logs"].append(f"[ERROR] {e}")
+        _push(job_id, {"stage": "failed", "current": job.get("current", 0), "total": job.get("total", 0), "message": str(e)})
+        _push(job_id, {"stage": "done", "current": 0, "total": 0, "message": "__DONE__"})
+    finally:
+        if active_job_id == job_id:
+            active_job_id = None

web/static/app.css

@@ -0,0 +1,975 @@
+:root {
+  --bg: #141517;
+  --surface: #1C1E21;
+  --surface-subtle: #25282D;
+  --border: #2B2E33;
+  --border-strong: #3E434A;
+  --text: #EAEBED;
+  --text-muted: #909399;
+  --primary: #0284C7;
+  --primary-hover: #0369A1;
+  --primary-fg: #FFFFFF;
+  --primary-subtle: rgba(2, 132, 199, 0.15);
+  --primary-accent: #38BDF8;
+  --btn-blue: #075985;
+  --btn-blue-hover: #0C4A6E;
+  --btn-blue-border: #0369A1;
+  --success: #34D399;
+  --success-bg: rgba(52, 211, 153, 0.1);
+  --success-border: #065F46;
+  --warning: #FBBF24;
+  --warning-bg: rgba(251, 191, 36, 0.1);
+  --warning-border: #78350F;
+  --danger: #FB7185;
+  --danger-bg: rgba(251, 113, 133, 0.1);
+  --danger-border: #881337;
+  --danger-hover: #F43F5E;
+  --log-bg: #0F1012;
+  --log-border: #2B2E33;
+  --log-text: #EAEBED;
+  --radius-sm: 4px;
+  --radius-md: 6px;
+  --radius-lg: 8px;
+  --radius-pill: 9999px;
+  --font-sans: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
+  --font-mono: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;
+  --transition-fast: 150ms ease;
+  --shadow-sm: 0 1px 2px rgba(0, 0, 0, 0.2);
+}
+
+*, *::before, *::after {
+  box-sizing: border-box;
+  margin: 0;
+  padding: 0;
+}
+
+body {
+  background-color: var(--bg);
+  color: var(--text);
+  font-family: var(--font-sans);
+  font-size: 15px;
+  line-height: 1.5;
+  min-height: 100vh;
+  display: flex;
+  flex-direction: column;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+}
+
+::selection {
+  background-color: var(--primary);
+  color: var(--primary-fg);
+}
+
+::-webkit-scrollbar {
+  width: 6px;
+  height: 6px;
+}
+::-webkit-scrollbar-thumb {
+  background: var(--border-strong);
+  border-radius: 3px;
+}
+::-webkit-scrollbar-track {
+  background: transparent;
+}
+
+:focus-visible {
+  outline: 2px solid var(--primary-accent);
+  outline-offset: 2px;
+}
+
+.app-header {
+  position: sticky;
+  top: 0;
+  z-index: 30;
+  background-color: var(--surface);
+  border-bottom: 1px solid var(--border);
+}
+
+.header-inner {
+  max-width: 56rem;
+  margin: 0 auto;
+  padding: 0.75rem 1rem;
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  gap: 1rem;
+}
+
+.brand-wrap {
+  display: flex;
+  align-items: center;
+  gap: 0.75rem;
+  text-decoration: none;
+  color: inherit;
+}
+
+.brand-icon {
+  width: 2rem;
+  height: 2rem;
+  border-radius: var(--radius-sm);
+  object-fit: contain;
+}
+
+.brand-title-line {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+}
+
+.brand-title {
+  font-size: 1.05rem;
+  font-weight: 600;
+  letter-spacing: -0.01em;
+  color: var(--text);
+}
+
+.brand-subtitle {
+  font-size: 0.8125rem;
+  color: var(--text-muted);
+}
+
+.header-badge {
+  font-size: 0.75rem;
+  font-weight: 600;
+  padding: 0.125rem 0.5rem;
+  border-radius: var(--radius-pill);
+  background-color: var(--success-bg);
+  color: var(--success);
+  border: 1px solid var(--success-border);
+}
+
+.header-badge.hidden {
+  display: none;
+}
+
+.header-actions {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+}
+
+.btn-header {
+  font-family: inherit;
+  font-size: 0.8125rem;
+  font-weight: 500;
+  color: var(--text-muted);
+  background-color: transparent;
+  border: 1px solid var(--border);
+  border-radius: var(--radius-md);
+  padding: 0.375rem 0.625rem;
+  min-height: 2rem;
+  display: inline-flex;
+  align-items: center;
+  gap: 0.375rem;
+  cursor: pointer;
+  text-decoration: none;
+  transition: all var(--transition-fast);
+}
+
+.btn-header:hover {
+  color: var(--text);
+  background-color: var(--surface-subtle);
+  border-color: var(--border-strong);
+}
+
+.btn-header.active {
+  color: var(--success);
+  border-color: var(--success);
+}
+
+.app-main {
+  max-width: 56rem;
+  margin: 0 auto;
+  width: 100%;
+  padding: 1.5rem 1rem;
+  display: flex;
+  flex-direction: column;
+  gap: 1.5rem;
+  flex: 1;
+}
+
+.card {
+  background-color: var(--surface);
+  border: 1px solid var(--border);
+  border-radius: var(--radius-lg);
+  padding: 1.25rem;
+  box-shadow: var(--shadow-sm);
+}
+
+.card-title {
+  font-size: 1rem;
+  font-weight: 600;
+  color: var(--text);
+  margin-bottom: 0.75rem;
+}
+
+.form-group {
+  margin-bottom: 1rem;
+}
+
+.form-label {
+  display: block;
+  font-size: 0.875rem;
+  font-weight: 500;
+  color: var(--text);
+  margin-bottom: 0.375rem;
+}
+
+.input-row {
+  display: flex;
+  gap: 0.5rem;
+}
+
+.input-text {
+  flex: 1;
+  font-family: inherit;
+  font-size: 0.9375rem;
+  background-color: var(--surface-subtle);
+  color: var(--text);
+  border: 1px solid var(--border);
+  border-radius: var(--radius-md);
+  padding: 0.625rem 0.875rem;
+  min-height: 2.75rem;
+  transition: border-color var(--transition-fast);
+}
+
+.input-text::placeholder {
+  color: var(--text-muted);
+  opacity: 0.8;
+}
+
+.input-text:hover {
+  border-color: var(--border-strong);
+}
+
+.input-text:focus {
+  border-color: var(--primary);
+  outline: none;
+}
+
+.btn-paste {
+  font-family: inherit;
+  font-size: 0.875rem;
+  font-weight: 500;
+  color: var(--text-muted);
+  background-color: var(--surface-subtle);
+  border: 1px solid var(--border);
+  border-radius: var(--radius-md);
+  padding: 0.625rem 0.875rem;
+  min-height: 2.75rem;
+  display: inline-flex;
+  align-items: center;
+  gap: 0.375rem;
+  cursor: pointer;
+  transition: all var(--transition-fast);
+  white-space: nowrap;
+}
+
+.btn-paste:hover {
+  color: var(--text);
+  border-color: var(--border-strong);
+}
+
+.cache-grid {
+  display: grid;
+  grid-template-columns: repeat(1, minmax(0, 1fr));
+  gap: 0.625rem;
+}
+
+@media (min-width: 640px) {
+  .cache-grid {
+    grid-template-columns: repeat(3, minmax(0, 1fr));
+  }
+}
+
+.cache-card {
+  display: flex;
+  align-items: flex-start;
+  gap: 0.75rem;
+  padding: 0.75rem;
+  border: 1px solid var(--border);
+  background-color: var(--surface-subtle);
+  border-radius: var(--radius-md);
+  cursor: pointer;
+  user-select: none;
+  transition: all var(--transition-fast);
+}
+
+.cache-card:hover {
+  border-color: var(--border-strong);
+}
+
+.cache-card:has(input:checked) {
+  border-color: var(--primary-accent);
+  background-color: var(--primary-subtle);
+}
+
+.cache-card input[type="radio"] {
+  margin-top: 0.1875rem;
+  accent-color: var(--primary);
+}
+
+.cache-card-title {
+  font-size: 0.875rem;
+  font-weight: 500;
+  color: var(--text);
+}
+
+.cache-card-desc {
+  font-size: 0.75rem;
+  color: var(--text-muted);
+  margin-top: 0.125rem;
+}
+
+.toggle-card {
+  display: flex;
+  align-items: flex-start;
+  gap: 0.75rem;
+  padding: 0.75rem;
+  border: 1px solid var(--border);
+  background-color: var(--surface-subtle);
+  border-radius: var(--radius-md);
+  cursor: pointer;
+  user-select: none;
+  transition: all var(--transition-fast);
+}
+
+.toggle-card:hover {
+  border-color: var(--border-strong);
+}
+
+.toggle-card:has(input:checked) {
+  border-color: var(--primary-accent);
+  background-color: var(--primary-subtle);
+}
+
+.toggle-card input[type="checkbox"] {
+  margin-top: 0.1875rem;
+  accent-color: var(--primary);
+}
+
+.badge-ape {
+  font-family: var(--font-mono);
+  font-size: 0.6875rem;
+  font-weight: 600;
+  text-transform: uppercase;
+  letter-spacing: 0.04em;
+  padding: 0.125rem 0.375rem;
+  border-radius: var(--radius-sm);
+  background-color: var(--primary-subtle);
+  color: var(--primary-accent);
+  border: 1px solid var(--primary);
+}
+
+.form-actions {
+  display: flex;
+  justify-content: flex-end;
+  padding-top: 0.25rem;
+}
+
+.btn-primary {
+  font-family: inherit;
+  font-size: 0.9375rem;
+  font-weight: 500;
+  color: var(--primary-fg);
+  background-color: var(--btn-blue);
+  border: 1px solid var(--btn-blue-border);
+  border-radius: var(--radius-md);
+  padding: 0.625rem 1.25rem;
+  min-height: 2.75rem;
+  display: inline-flex;
+  align-items: center;
+  justify-content: center;
+  gap: 0.5rem;
+  cursor: pointer;
+  transition: all var(--transition-fast);
+  box-shadow: var(--shadow-sm);
+  width: 100%;
+}
+
+@media (min-width: 640px) {
+  .btn-primary {
+    width: auto;
+  }
+}
+
+.btn-primary:hover:not(:disabled) {
+  background-color: var(--btn-blue-hover);
+  border-color: var(--btn-blue);
+}
+
+.btn-primary:disabled {
+  opacity: 0.6;
+  cursor: not-allowed;
+}
+
+.monitor-section.hidden {
+  display: none;
+}
+
+.monitor-header {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  margin-bottom: 0.75rem;
+}
+
+.monitor-title-group {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+}
+
+.status-dot {
+  width: 0.625rem;
+  height: 0.625rem;
+  border-radius: 50%;
+  background-color: var(--success);
+  flex-shrink: 0;
+}
+
+.status-dot.dot-emerald { background-color: var(--success); }
+.status-dot.dot-amber { background-color: var(--warning); }
+.status-dot.dot-rose { background-color: var(--danger); }
+
+.monitor-actions {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+}
+
+.btn-stop {
+  font-family: inherit;
+  font-size: 0.8125rem;
+  font-weight: 500;
+  color: var(--danger);
+  background-color: var(--danger-bg);
+  border: 1px solid var(--danger-border);
+  border-radius: var(--radius-md);
+  padding: 0.25rem 0.625rem;
+  cursor: pointer;
+  transition: all var(--transition-fast);
+}
+
+.btn-stop:hover:not(:disabled) {
+  background-color: var(--danger);
+  color: #FFFFFF;
+}
+
+.btn-stop.hidden {
+  display: none;
+}
+
+.badge-status {
+  font-family: var(--font-mono);
+  font-size: 0.6875rem;
+  font-weight: 600;
+  text-transform: uppercase;
+  padding: 0.125rem 0.5rem;
+  border-radius: var(--radius-sm);
+  border: 1px solid var(--border);
+  background-color: var(--surface-subtle);
+  color: var(--text);
+}
+
+.stage-row {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  font-size: 0.875rem;
+  gap: 0.75rem;
+  margin-bottom: 0.5rem;
+}
+
+.stage-pill {
+  font-family: var(--font-mono);
+  font-size: 0.75rem;
+  font-weight: 600;
+  text-transform: uppercase;
+  padding: 0.125rem 0.5rem;
+  border-radius: var(--radius-sm);
+  background-color: var(--surface-subtle);
+  border: 1px solid var(--border);
+  color: var(--text);
+  flex-shrink: 0;
+}
+
+.stage-message {
+  color: var(--text);
+  font-weight: 500;
+  white-space: nowrap;
+  overflow: hidden;
+  text-overflow: ellipsis;
+}
+
+.stage-counter {
+  font-family: var(--font-mono);
+  font-size: 0.8125rem;
+  color: var(--text-muted);
+  flex-shrink: 0;
+}
+
+.progress-track {
+  height: 0.5rem;
+  border-radius: var(--radius-pill);
+  background-color: var(--surface-subtle);
+  border: 1px solid var(--border);
+  overflow: hidden;
+  margin-bottom: 0.75rem;
+}
+
+.progress-fill {
+  height: 100%;
+  background-color: var(--primary);
+  width: 0%;
+  transition: width 200ms ease;
+}
+
+.log-meta-row {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  font-size: 0.8125rem;
+  color: var(--text-muted);
+  margin-bottom: 0.375rem;
+}
+
+.log-actions {
+  display: flex;
+  align-items: center;
+  gap: 0.75rem;
+}
+
+.log-action-btn {
+  background: none;
+  border: none;
+  font-family: inherit;
+  font-size: 0.8125rem;
+  color: var(--text-muted);
+  cursor: pointer;
+  transition: color var(--transition-fast);
+}
+
+.log-action-btn:hover {
+  color: var(--text);
+}
+
+.log-box {
+  height: 10rem;
+  overflow: auto;
+  border-radius: var(--radius-md);
+  background-color: var(--log-bg);
+  border: 1px solid var(--log-border);
+  padding: 0.75rem;
+  font-family: var(--font-mono);
+  font-size: 0.8125rem;
+  color: var(--log-text);
+  white-space: pre-wrap;
+  line-height: 1.6;
+}
+
+.log-box.expanded {
+  height: 24rem;
+}
+
+.result-bar {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  margin-top: 0.75rem;
+}
+
+.result-bar.hidden {
+  display: none;
+}
+
+.btn-secondary {
+  font-family: inherit;
+  font-size: 0.875rem;
+  font-weight: 500;
+  color: var(--text-muted);
+  background-color: transparent;
+  border: 1px solid var(--border);
+  border-radius: var(--radius-md);
+  padding: 0.625rem 1rem;
+  min-height: 2.75rem;
+  cursor: pointer;
+  transition: all var(--transition-fast);
+}
+
+.btn-secondary:hover {
+  color: var(--text);
+  background-color: var(--surface-subtle);
+  border-color: var(--border-strong);
+}
+
+.library-header {
+  display: flex;
+  flex-direction: column;
+  gap: 0.75rem;
+  margin-bottom: 1rem;
+}
+
+@media (min-width: 640px) {
+  .library-header {
+    flex-direction: row;
+    align-items: center;
+    justify-content: space-between;
+    gap: 1.5rem;
+  }
+}
+
+.library-title-group {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  flex-shrink: 0;
+}
+
+.library-counter {
+  font-family: var(--font-mono);
+  font-size: 0.8125rem;
+  color: var(--text-muted);
+}
+
+.library-controls {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  flex: 1;
+}
+
+.search-wrap {
+  position: relative;
+  flex: 1;
+  display: flex;
+}
+
+.search-wrap .input-text {
+  width: 100%;
+}
+
+.book-shelf {
+  display: flex;
+  flex-direction: column;
+  gap: 0.75rem;
+}
+
+.book-card {
+  border: 1px solid var(--border);
+  background-color: var(--surface-subtle);
+  border-radius: var(--radius-md);
+  overflow: hidden;
+  transition: border-color var(--transition-fast);
+}
+
+.book-card:hover {
+  border-color: var(--border-strong);
+}
+
+.book-edition-row {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  gap: 0.75rem;
+  padding: 0.75rem 1rem;
+}
+
+.book-edition-row + .book-edition-row {
+  border-top: 1px solid var(--border);
+}
+
+.book-edition-row.fixed-edition {
+  background-color: var(--primary-subtle);
+}
+
+.book-info {
+  display: flex;
+  align-items: center;
+  gap: 0.75rem;
+  min-width: 0;
+  flex: 1;
+}
+
+.book-icon {
+  width: 1.125rem;
+  height: 1.125rem;
+  color: var(--text-muted);
+  flex-shrink: 0;
+}
+
+.fixed-edition .book-icon {
+  color: var(--primary-accent);
+}
+
+.book-meta-block {
+  min-width: 0;
+  flex: 1;
+}
+
+.book-title-line {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  flex-wrap: wrap;
+}
+
+.book-title {
+  font-size: 0.9375rem;
+  font-weight: 600;
+  color: var(--text);
+  white-space: nowrap;
+  overflow: hidden;
+  text-overflow: ellipsis;
+  text-transform: capitalize;
+}
+
+.badge-fixed {
+  font-family: var(--font-mono);
+  font-size: 0.6875rem;
+  font-weight: 600;
+  text-transform: uppercase;
+  padding: 0.0625rem 0.375rem;
+  border-radius: var(--radius-sm);
+  background-color: var(--primary);
+  color: var(--primary-fg);
+}
+
+.book-specs {
+  font-family: var(--font-mono);
+  font-size: 0.75rem;
+  color: var(--text-muted);
+  margin-top: 0.125rem;
+}
+
+.book-actions {
+  display: flex;
+  align-items: center;
+  gap: 0.375rem;
+  flex-shrink: 0;
+}
+
+.btn-book-action {
+  font-family: inherit;
+  font-size: 0.8125rem;
+  font-weight: 500;
+  padding: 0.375rem 0.625rem;
+  min-height: 2rem;
+  border-radius: var(--radius-sm);
+  cursor: pointer;
+  text-decoration: none;
+  display: inline-flex;
+  align-items: center;
+  justify-content: center;
+  transition: all var(--transition-fast);
+}
+
+.btn-book-download {
+  color: var(--primary-fg);
+  background-color: var(--btn-blue);
+  border: 1px solid var(--btn-blue-border);
+}
+
+.btn-book-download:hover {
+  background-color: var(--btn-blue-hover);
+}
+
+.btn-book-fix {
+  color: var(--text-muted);
+  background-color: transparent;
+  border: 1px solid var(--border);
+}
+
+.btn-book-fix:hover {
+  color: var(--primary-accent);
+  border-color: var(--primary-accent);
+}
+
+.btn-book-delete {
+  color: var(--text-muted);
+  background-color: transparent;
+  border: 1px solid var(--border);
+}
+
+.btn-book-delete:hover {
+  color: var(--danger);
+  border-color: var(--danger-border);
+}
+
+.delete-confirm-box {
+  display: inline-flex;
+  align-items: center;
+  gap: 0.25rem;
+  background-color: var(--danger-bg);
+  border: 1px solid var(--danger-border);
+  border-radius: var(--radius-sm);
+  padding: 0.125rem;
+}
+
+.btn-confirm-delete {
+  font-family: inherit;
+  font-size: 0.75rem;
+  font-weight: 600;
+  padding: 0.25rem 0.5rem;
+  border-radius: var(--radius-sm);
+  background-color: var(--danger);
+  color: #FFFFFF;
+  border: none;
+  cursor: pointer;
+}
+
+.btn-confirm-delete:hover {
+  background-color: var(--danger-hover);
+}
+
+.btn-cancel-delete {
+  font-family: inherit;
+  font-size: 0.75rem;
+  padding: 0.25rem 0.375rem;
+  background: transparent;
+  border: none;
+  color: var(--text-muted);
+  cursor: pointer;
+}
+
+.btn-cancel-delete:hover {
+  color: var(--text);
+}
+
+.empty-state {
+  text-align: center;
+  padding: 2.5rem 1rem;
+  color: var(--text-muted);
+  font-size: 0.875rem;
+}
+
+.empty-state.hidden {
+  display: none;
+}
+
+.guide-details {
+  border: 1px solid var(--border);
+  background-color: var(--surface);
+  border-radius: var(--radius-lg);
+  padding: 0.75rem 1rem;
+}
+
+.guide-summary {
+  font-size: 0.875rem;
+  font-weight: 600;
+  color: var(--text);
+  cursor: pointer;
+  user-select: none;
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+}
+
+.guide-grid {
+  display: grid;
+  grid-template-columns: repeat(1, minmax(0, 1fr));
+  gap: 0.625rem;
+  margin-top: 0.75rem;
+}
+
+@media (min-width: 640px) {
+  .guide-grid {
+    grid-template-columns: repeat(2, minmax(0, 1fr));
+  }
+}
+
+@media (min-width: 1024px) {
+  .guide-grid {
+    grid-template-columns: repeat(4, minmax(0, 1fr));
+  }
+}
+
+.guide-step {
+  padding: 0.75rem;
+  border-radius: var(--radius-md);
+  background-color: var(--surface-subtle);
+  border: 1px solid var(--border);
+  font-size: 0.75rem;
+  color: var(--text-muted);
+}
+
+.guide-step-title {
+  font-size: 0.8125rem;
+  font-weight: 600;
+  color: var(--text);
+  margin-bottom: 0.25rem;
+}
+
+.app-footer {
+  border-top: 1px solid var(--border);
+  padding: 1rem;
+  text-align: center;
+  font-size: 0.8125rem;
+  color: var(--text-muted);
+  margin-top: auto;
+}
+
+.toast-container {
+  position: fixed;
+  bottom: 1rem;
+  right: 1rem;
+  z-index: 50;
+  display: flex;
+  flex-direction: column;
+  gap: 0.5rem;
+  pointer-events: none;
+}
+
+.toast {
+  pointer-events: auto;
+  border-radius: var(--radius-md);
+  border: 1px solid var(--border);
+  background-color: var(--surface);
+  color: var(--text);
+  padding: 0.625rem 0.875rem;
+  font-size: 0.875rem;
+  box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.3), 0 2px 4px -2px rgba(0, 0, 0, 0.2);
+  display: flex;
+  align-items: center;
+  gap: 0.625rem;
+  transition: opacity 150ms ease, transform 150ms ease;
+  opacity: 0;
+  transform: translateY(4px);
+}
+
+.toast.show {
+  opacity: 1;
+  transform: translateY(0);
+}
+
+.toast.toast-success {
+  border-color: var(--success);
+  background-color: var(--surface);
+  color: var(--success);
+}
+
+.toast.toast-error {
+  border-color: var(--danger);
+  background-color: var(--surface);
+  color: var(--danger);
+}
+
+.toast.toast-info {
+  border-color: var(--border-strong);
+}
+
+.spinner {
+  width: 1rem;
+  height: 1rem;
+  animation: spin 1s linear infinite;
+}
+
+.spinner.hidden {
+  display: none;
+}
+
+@keyframes spin {
+  from { transform: rotate(0deg); }
+  to { transform: rotate(360deg); }
+}

web/templates/index.html

@@ -0,0 +1,756 @@
+<!DOCTYPE html>
+<html lang="en" class="dark">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0">
+<title>Silksoup: Novel to EPUB</title>
+<link rel="icon" type="image/x-icon" href="/static/favicon.ico">
+<link rel="icon" type="image/png" href="/static/icon.png">
+<link rel="apple-touch-icon" href="/static/icon.png">
+<link rel="stylesheet" href="/static/app.css">
+</head>
+<body>
+<header class="app-header">
+  <div class="header-inner">
+    <div class="brand-wrap">
+      <img src="/static/icon.png" alt="Silksoup" class="brand-icon">
+      <div>
+        <div class="brand-title-line">
+          <span class="brand-title">Silksoup</span>
+          <span id="headerBadge" class="header-badge hidden">Active</span>
+        </div>
+        <p class="brand-subtitle">Novel to EPUB pipeline</p>
+      </div>
+    </div>
+    <div class="header-actions">
+      <button id="notifyBtn" onclick="toggleNotifications()" type="button" aria-label="Desktop notifications" class="btn-header">
+        <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+          <path d="M6 8a6 6 0 0 1 12 0c0 7 3 9 3 9H3s3-2 3-9"/>
+          <path d="M10.3 21a1.94 1.94 0 0 0 3.4 0"/>
+        </svg>
+        <span id="notifyText">Notify</span>
+      </button>
+      <a href="/api/epubs" target="_blank" rel="noopener" class="btn-header">
+        <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+          <polyline points="16 18 22 12 16 6"/>
+          <polyline points="8 6 2 12 8 18"/>
+        </svg>
+        <span>API</span>
+      </a>
+    </div>
+  </div>
+</header>
+
+<main class="app-main">
+  <section class="card" aria-labelledby="headingScrape">
+    <h2 id="headingScrape" class="card-title">Scrape Novel</h2>
+    <form id="jobForm">
+      <div class="form-group">
+        <label for="urlInput" class="form-label">Novel URL</label>
+        <div class="input-row">
+          <input id="urlInput" type="url" required placeholder="https://kakuyomu.jp/works/... or novel index URL" class="input-text">
+          <button type="button" onclick="pasteClipboard()" class="btn-paste">
+            <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+              <rect width="8" height="4" x="8" y="2" rx="1" ry="1"/>
+              <path d="M16 4h2a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2"/>
+            </svg>
+            <span>Paste</span>
+          </button>
+        </div>
+      </div>
+      <div class="form-group">
+        <span class="form-label">Cache Mode</span>
+        <div class="cache-grid" role="radiogroup" aria-label="Cache Mode">
+          <label class="cache-card">
+            <input type="radio" name="refreshMode" value="none" checked>
+            <div>
+              <div class="cache-card-title">Full Cache</div>
+              <div class="cache-card-desc">Reuse HTML and AI results</div>
+            </div>
+          </label>
+          <label class="cache-card">
+            <input type="radio" name="refreshMode" value="llm">
+            <div>
+              <div class="cache-card-title">Re-run AI</div>
+              <div class="cache-card-desc">Reuse HTML, re-sanitize chapters</div>
+            </div>
+          </label>
+          <label class="cache-card">
+            <input type="radio" name="refreshMode" value="all">
+            <div>
+              <div class="cache-card-title">Full Refresh</div>
+              <div class="cache-card-desc">Bypass cache and fetch fresh</div>
+            </div>
+          </label>
+        </div>
+      </div>
+      <div class="form-group">
+        <label class="toggle-card">
+          <input id="fixCheck" type="checkbox">
+          <div>
+            <div style="display: flex; align-items: center; gap: 0.5rem;">
+              <span class="cache-card-title">Polished Edition (Automated Post-Editing)</span>
+              <span class="badge-ape">APE</span>
+            </div>
+            <div class="cache-card-desc">Repair machine translation phrasing, pronoun confusion, and format translator notes</div>
+          </div>
+        </label>
+      </div>
+      <div class="form-actions">
+        <button id="startBtn" type="submit" class="btn-primary">
+          <svg id="btnSpinner" class="spinner hidden" viewBox="0 0 24 24" fill="none">
+            <circle opacity="0.25" cx="12" cy="12" r="10" stroke="currentColor" stroke-width="4"></circle>
+            <path opacity="0.75" fill="currentColor" d="M4 12a8 8 0 018-8v4a4 4 0 00-4 4H4z"></path>
+          </svg>
+          <span id="btnText">Start Scraping</span>
+        </button>
+      </div>
+    </form>
+  </section>
+
+  <section id="activeJob" class="card monitor-section hidden" aria-labelledby="headingMonitor" role="region">
+    <div class="monitor-header">
+      <div class="monitor-title-group">
+        <span id="statusPulse" class="status-dot dot-emerald"></span>
+        <h2 id="headingMonitor" class="card-title" style="margin-bottom: 0;">Task Monitor</h2>
+      </div>
+      <div class="monitor-actions">
+        <button id="cancelJobBtn" type="button" onclick="requestCancelJob()" class="btn-stop">Stop Process</button>
+        <span id="jobStatusBadge" class="badge-status">idle</span>
+      </div>
+    </div>
+
+    <div>
+      <div class="stage-row">
+        <div style="display: flex; align-items: center; gap: 0.5rem; min-width: 0;">
+          <span id="jobStage" class="stage-pill">init</span>
+          <span id="jobMessage" class="stage-message">Starting...</span>
+        </div>
+        <span id="jobCounter" class="stage-counter">0 / 0</span>
+      </div>
+
+      <div class="progress-track" role="progressbar" id="progressTrack" aria-valuenow="0" aria-valuemin="0" aria-valuemax="100">
+        <div id="jobProgress" class="progress-fill"></div>
+      </div>
+
+      <div class="log-meta-row">
+        <span>Execution Log</span>
+        <div class="log-actions">
+          <label style="display: inline-flex; align-items: center; gap: 0.375rem; cursor: pointer; user-select: none;">
+            <input id="autoScrollCheck" type="checkbox" checked style="accent-color: var(--primary);">
+            <span>Auto-scroll</span>
+          </label>
+          <button type="button" onclick="clearLogs()" class="log-action-btn">Clear</button>
+          <button type="button" onclick="toggleLogHeight()" id="expandLogBtn" class="log-action-btn">Expand</button>
+        </div>
+      </div>
+
+      <div id="jobLog" class="log-box" role="log" aria-live="polite"></div>
+
+      <div id="jobResult" class="result-bar hidden">
+        <a id="downloadLink" href="#" class="btn-primary" style="text-decoration: none;">
+          <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+            <path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
+            <polyline points="7 10 12 15 17 10"/>
+            <line x1="12" y1="15" x2="12" y2="3"/>
+          </svg>
+          <span>Download EPUB</span>
+        </a>
+        <button type="button" onclick="dismissActiveJob()" class="btn-secondary">Dismiss</button>
+      </div>
+    </div>
+  </section>
+
+  <section class="card" aria-labelledby="headingLibrary">
+    <div class="library-header">
+      <div class="library-title-group">
+        <h2 id="headingLibrary" class="card-title" style="margin-bottom: 0;">Library</h2>
+        <span id="epubCount" class="library-counter"></span>
+      </div>
+      <div class="library-controls">
+        <div class="search-wrap">
+          <input id="searchEpub" type="text" oninput="filterEpubs()" placeholder="Filter books..." class="input-text" style="min-height: 2.25rem; font-size: 0.875rem;">
+        </div>
+        <button type="button" onclick="loadEpubs()" class="btn-header" style="min-height: 2.25rem;">Refresh</button>
+      </div>
+    </div>
+
+    <div id="epubList" class="book-shelf"></div>
+    <div id="epubEmpty" class="empty-state hidden">
+      No EPUBs found. Scrape a novel to populate your library.
+    </div>
+  </section>
+
+  <details class="guide-details">
+    <summary class="guide-summary">
+      <span>Workflow Pipeline Guide</span>
+      <span style="font-size: 0.75rem; color: var(--text-muted);">Details</span>
+    </summary>
+    <div class="guide-grid">
+      <div class="guide-step">
+        <div class="guide-step-title">1. Ingest & Cache</div>
+        Fetches novel HTML via anti-bot client into local database.
+      </div>
+      <div class="guide-step">
+        <div class="guide-step-title">2. Parse Index</div>
+        LLM extracts chronological volumes and chapter URLs.
+      </div>
+      <div class="guide-step">
+        <div class="guide-step-title">3. Sanitize & APE</div>
+        Fixes machine translation flaws, entities, and formats notes into blockquotes.
+      </div>
+      <div class="guide-step">
+        <div class="guide-step-title">4. Build EPUB</div>
+        Downloads images, compresses assets, and compiles book.
+      </div>
+    </div>
+  </details>
+</main>
+
+<div id="toastContainer" class="toast-container" role="region" aria-live="polite"></div>
+
+<footer class="app-footer">
+  Silksoup: Novel to EPUB Pipeline
+</footer>
+
+<script>
+const $ = s => document.querySelector(s);
+let currentJobId = localStorage.getItem('silksoup_active_job') || null;
+let evtSource = null;
+let allEpubs = [];
+let originalTitle = document.title;
+
+function updateNotifyButton() {
+  if (Notification.permission === 'granted') {
+    $('#notifyBtn').classList.add('active');
+    $('#notifyText').textContent = 'Notifications on';
+  }
+}
+
+async function toggleNotifications() {
+  if (Notification.permission === 'default') {
+    const res = await Notification.requestPermission();
+    if (res === 'granted') {
+      showToast('Notifications enabled', 'success');
+      updateNotifyButton();
+    }
+  } else if (Notification.permission === 'denied') {
+    showToast('Notifications blocked in browser settings', 'error');
+  } else {
+    showToast('Notifications already enabled', 'info');
+  }
+}
+
+function notify(title, body) {
+  if (Notification.permission === 'granted' && document.hidden) {
+    new Notification(title, { body, icon: '/static/icon.png' });
+  }
+}
+
+function showToast(message, type = 'info') {
+  const container = $('#toastContainer');
+  const toast = document.createElement('div');
+  const typeClass = type === 'error' ? 'toast-error' :
+                    type === 'success' ? 'toast-success' : 'toast-info';
+  toast.className = `toast ${typeClass}`;
+  toast.textContent = message;
+  container.appendChild(toast);
+  requestAnimationFrame(() => {
+    toast.classList.add('show');
+  });
+  setTimeout(() => {
+    toast.classList.remove('show');
+    setTimeout(() => toast.remove(), 150);
+  }, 4000);
+}
+
+async function pasteClipboard() {
+  try {
+    const text = await navigator.clipboard.readText();
+    if (text) {
+      $('#urlInput').value = text.trim();
+      showToast('Pasted URL from clipboard', 'info');
+    }
+  } catch(e) {
+    showToast('Clipboard access denied', 'error');
+  }
+}
+
+function setBtnLoading(loading) {
+  const btn = $('#startBtn');
+  const spinner = $('#btnSpinner');
+  const text = $('#btnText');
+  if (loading) {
+    btn.disabled = true;
+    spinner.classList.remove('hidden');
+    text.textContent = 'Submitting...';
+  } else {
+    btn.disabled = false;
+    spinner.classList.add('hidden');
+    text.textContent = 'Start Scraping';
+  }
+}
+
+function setFormLocked(locked) {
+  $('#urlInput').disabled = locked;
+  document.querySelectorAll('input[name="refreshMode"]').forEach(r => r.disabled = locked);
+  const fixCheck = $('#fixCheck');
+  if (fixCheck) fixCheck.disabled = locked;
+  $('#startBtn').disabled = locked;
+}
+
+async function submitJob() {
+  const url = $('#urlInput').value.trim();
+  if (!url) {
+    showToast('Please enter a novel URL', 'error');
+    return;
+  }
+  setBtnLoading(true);
+  try {
+    const selectedMode = document.querySelector('input[name="refreshMode"]:checked');
+    const refresh_mode = selectedMode ? selectedMode.value : 'none';
+    const fix = $('#fixCheck') ? $('#fixCheck').checked : false;
+    const res = await fetch('/api/jobs', {
+      method: 'POST',
+      headers: {'Content-Type': 'application/json'},
+      body: JSON.stringify({ url, refresh_mode, fix })
+    });
+    if (res.status === 409) {
+      const j = await res.json();
+      showToast(j.detail || 'Another novel conversion is currently in progress', 'error');
+      await syncActiveJob();
+      return;
+    }
+    if (!res.ok) {
+      const j = await res.json();
+      throw new Error(j.detail || 'Failed to submit job');
+    }
+    const data = await res.json();
+    currentJobId = data.job_id;
+    localStorage.setItem('silksoup_active_job', currentJobId);
+    showActive();
+    connectSSE(currentJobId);
+    showToast('Job started successfully', 'success');
+  } catch(e) {
+    showToast(e.message, 'error');
+  } finally {
+    setBtnLoading(false);
+  }
+}
+
+$('#jobForm').addEventListener('submit', (e) => {
+  e.preventDefault();
+  submitJob();
+});
+
+function showActive(jobData = null) {
+  setFormLocked(true);
+  $('#activeJob').classList.remove('hidden');
+  $('#headerBadge').classList.remove('hidden');
+  $('#cancelJobBtn').classList.remove('hidden');
+  if (!jobData) {
+    $('#jobLog').textContent = '';
+    $('#jobProgress').style.width = '0%';
+    $('#progressTrack').setAttribute('aria-valuenow', '0');
+    $('#jobCounter').textContent = '0 / 0';
+    $('#jobStage').textContent = 'queued';
+    $('#jobStatusBadge').textContent = 'running';
+    $('#jobResult').classList.add('hidden');
+    $('#statusPulse').className = 'status-dot dot-emerald';
+  } else {
+    $('#jobStage').textContent = jobData.stage || 'running';
+    $('#jobMessage').textContent = jobData.message || '';
+    $('#jobCounter').textContent = `${jobData.current || 0} / ${jobData.total || 0}`;
+    const pct = jobData.percentage != null ? jobData.percentage :
+      (jobData.total ? Math.round((jobData.current / jobData.total) * 100) : 0);
+    $('#jobProgress').style.width = pct + '%';
+    $('#progressTrack').setAttribute('aria-valuenow', String(pct));
+    $('#jobStatusBadge').textContent = jobData.status || 'running';
+    $('#statusPulse').className = 'status-dot dot-emerald';
+    if (jobData.logs && jobData.logs.length) {
+      $('#jobLog').textContent = jobData.logs.join('\n') + '\n';
+      $('#jobLog').scrollTop = $('#jobLog').scrollHeight;
+    }
+    if (jobData.status === 'completed') {
+      setFormLocked(false);
+      $('#cancelJobBtn').classList.add('hidden');
+      if (jobData.result_path) {
+        const name = jobData.result_path.split(/[\\/]/).pop();
+        $('#downloadLink').href = `/api/epubs/${encodeURIComponent(name)}`;
+        $('#downloadLink').querySelector('span').textContent = `Download ${name}`;
+        $('#jobResult').classList.remove('hidden');
+      }
+    } else if (jobData.status === 'cancelled' || jobData.status === 'failed') {
+      setFormLocked(false);
+      $('#cancelJobBtn').classList.add('hidden');
+      if (jobData.status === 'cancelled') {
+        $('#statusPulse').className = 'status-dot dot-amber';
+      } else {
+        $('#statusPulse').className = 'status-dot dot-rose';
+      }
+    }
+  }
+  $('#activeJob').scrollIntoView({ behavior: 'smooth', block: 'nearest' });
+}
+
+async function requestCancelJob() {
+  if (!currentJobId) return;
+  const btn = $('#cancelJobBtn');
+  btn.disabled = true;
+  btn.textContent = 'Stopping...';
+  try {
+    const res = await fetch(`/api/jobs/${currentJobId}/cancel`, { method: 'POST' });
+    if (res.ok) {
+      showToast('Process cancellation requested', 'info');
+      setFormLocked(false);
+    }
+  } catch(e) {
+    showToast('Failed to cancel job', 'error');
+  } finally {
+    btn.disabled = false;
+    btn.textContent = 'Stop Process';
+  }
+}
+
+function dismissActiveJob() {
+  localStorage.removeItem('silksoup_active_job');
+  currentJobId = null;
+  if (evtSource) evtSource.close();
+  $('#activeJob').classList.add('hidden');
+  $('#headerBadge').classList.add('hidden');
+  setFormLocked(false);
+  document.title = originalTitle;
+}
+
+function connectSSE(jobId) {
+  if (evtSource) evtSource.close();
+  evtSource = new EventSource(`/api/jobs/${jobId}/events`);
+
+  evtSource.onmessage = (e) => {
+    if (e.data.startsWith(':')) return;
+    const evt = JSON.parse(e.data);
+    if (evt.message === '__DONE__') {
+      evtSource.close();
+      refreshJob();
+      loadEpubs();
+      setFormLocked(false);
+      return;
+    }
+    const pct = evt.total ? Math.round(evt.current / evt.total * 100) : 0;
+    $('#jobStage').textContent = evt.stage;
+    $('#jobMessage').textContent = evt.message || '';
+    $('#jobCounter').textContent = `${evt.current} / ${evt.total}`;
+    $('#jobProgress').style.width = pct + '%';
+    $('#progressTrack').setAttribute('aria-valuenow', String(pct));
+
+    if (evt.stage === 'scraping' || evt.stage === 'cleaning' || evt.stage === 'fixing' || evt.stage === 'epub') {
+      document.title = `(${pct}%) Silksoup`;
+    }
+
+    if (evt.stage === 'failed') {
+      $('#jobStatusBadge').textContent = 'failed';
+      $('#statusPulse').className = 'status-dot dot-rose';
+      $('#cancelJobBtn').classList.add('hidden');
+      setFormLocked(false);
+      loadEpubs();
+      document.title = 'Failed: Silksoup';
+      showToast(`Job failed: ${evt.message}`, 'error');
+    }
+    if (evt.stage === 'cancelled') {
+      $('#jobStatusBadge').textContent = 'cancelled';
+      $('#statusPulse').className = 'status-dot dot-amber';
+      $('#cancelJobBtn').classList.add('hidden');
+      setFormLocked(false);
+      loadEpubs();
+      document.title = 'Silksoup';
+      showToast('Process stopped. Cached chapters remain safe.', 'info');
+    }
+    if (evt.stage === 'completed') {
+      $('#jobStatusBadge').textContent = 'completed';
+      $('#statusPulse').className = 'status-dot dot-emerald';
+      $('#cancelJobBtn').classList.add('hidden');
+      setFormLocked(false);
+      loadEpubs();
+      document.title = 'Completed: Silksoup';
+      notify('Silksoup EPUB Ready', evt.message || 'Book compiled successfully.');
+      showToast('EPUB compiled successfully', 'success');
+    }
+
+    if (evt.message && evt.message !== '__DONE__') {
+      const log = $('#jobLog');
+      log.textContent += `[${evt.stage}] ${evt.message}\n`;
+      if ($('#autoScrollCheck').checked) {
+        log.scrollTop = log.scrollHeight;
+      }
+    }
+  };
+
+  evtSource.onerror = () => {};
+}
+
+async function refreshJob() {
+  if (!currentJobId) return;
+  const res = await fetch(`/api/jobs/${currentJobId}`);
+  if (!res.ok) return;
+  const j = await res.json();
+  $('#jobStatusBadge').textContent = j.status;
+  if (j.status === 'completed') {
+    $('#statusPulse').className = 'status-dot dot-emerald';
+    $('#cancelJobBtn').classList.add('hidden');
+    if (j.result_path) {
+      const name = j.result_path.split(/[\\/]/).pop();
+      $('#downloadLink').href = `/api/epubs/${encodeURIComponent(name)}`;
+      $('#downloadLink').querySelector('span').textContent = `Download ${name}`;
+      $('#jobResult').classList.remove('hidden');
+    }
+  }
+  if (j.status === 'cancelled') {
+    $('#statusPulse').className = 'status-dot dot-amber';
+    $('#cancelJobBtn').classList.add('hidden');
+  }
+  if (j.status === 'failed') {
+    $('#statusPulse').className = 'status-dot dot-rose';
+    $('#cancelJobBtn').classList.add('hidden');
+    $('#jobLog').textContent += `\n[ERROR] ${j.error || j.message}\n`;
+  }
+}
+
+function clearLogs() {
+  $('#jobLog').textContent = '';
+}
+
+let isLogExpanded = false;
+function toggleLogHeight() {
+  const log = $('#jobLog');
+  const btn = $('#expandLogBtn');
+  if (isLogExpanded) {
+    log.classList.remove('expanded');
+    btn.textContent = 'Expand';
+    isLogExpanded = false;
+  } else {
+    log.classList.add('expanded');
+    btn.textContent = 'Collapse';
+    isLogExpanded = true;
+  }
+}
+
+async function loadEpubs() {
+  try {
+    const res = await fetch('/api/epubs');
+    const data = await res.json();
+    allEpubs = data.epubs || [];
+    $('#epubCount').textContent = allEpubs.length ? `(${allEpubs.length})` : '';
+    renderEpubs(allEpubs);
+  } catch(e) {
+    showToast('Failed to load library', 'error');
+  }
+}
+
+function filterEpubs() {
+  const query = $('#searchEpub').value.toLowerCase().trim();
+  if (!query) {
+    renderEpubs(allEpubs);
+    return;
+  }
+  const filtered = allEpubs.filter(f => f.name.toLowerCase().includes(query));
+  renderEpubs(filtered, true);
+}
+
+async function triggerFix(encodedName) {
+  const name = decodeURIComponent(encodedName);
+  try {
+    const res = await fetch(`/api/epubs/${encodeURIComponent(name)}/fix`, { method: 'POST' });
+    if (res.status === 409) {
+      const j = await res.json();
+      showToast(j.detail || 'Another novel conversion is currently in progress', 'error');
+      await syncActiveJob();
+      return;
+    }
+    if (!res.ok) {
+      const j = await res.json();
+      throw new Error(j.detail || 'Failed to start fix job');
+    }
+    const data = await res.json();
+    currentJobId = data.job_id;
+    localStorage.setItem('silksoup_active_job', currentJobId);
+    showActive();
+    connectSSE(currentJobId);
+    showToast('Fix process started for ' + name, 'success');
+  } catch(e) {
+    showToast(e.message, 'error');
+  }
+}
+
+function renderEpubs(list, isFiltered = false) {
+  const container = $('#epubList');
+  container.innerHTML = '';
+  if (!list.length) {
+    $('#epubEmpty').textContent = isFiltered ?
+      'No books match your search filter.' :
+      'No EPUBs found. Scrape a novel to populate your library.';
+    $('#epubEmpty').classList.remove('hidden');
+    return;
+  }
+  $('#epubEmpty').classList.add('hidden');
+
+  const groups = {};
+  for (const f of list) {
+    const key = f.base_key || f.name.replace(/(-fixed)?\.epub$/i, '').trim().toLowerCase();
+    if (!groups[key]) groups[key] = { regular: null, fixed: null };
+    if (f.is_fixed) {
+      groups[key].fixed = f;
+    } else {
+      groups[key].regular = f;
+    }
+  }
+
+  for (const key of Object.keys(groups)) {
+    const g = groups[key];
+    const groupCard = document.createElement('div');
+    groupCard.className = 'book-card';
+
+    if (g.regular) {
+      const f = g.regular;
+      const cleanTitle = f.name.replace(/\.epub$/i, '').replace(/[-_]/g, ' ');
+      const row = document.createElement('div');
+      row.className = 'book-edition-row';
+
+      row.innerHTML = `
+        <div class="book-info">
+          <svg class="book-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+            <path d="M4 19.5v-15A2.5 2.5 0 0 1 6.5 2H20v20H6.5a2.5 2.5 0 0 1-2.5-2.5Z"/>
+            <path d="M6 6h10"/>
+            <path d="M6 10h10"/>
+          </svg>
+          <div class="book-meta-block">
+            <div class="book-title-line">
+              <span class="book-title" title="${cleanTitle}">${cleanTitle}</span>
+            </div>
+            <div class="book-specs">${f.size_human} • ${new Date(f.mtime * 1000).toLocaleDateString()} ${new Date(f.mtime * 1000).toLocaleTimeString([], {hour: '2-digit', minute: '2-digit'})}</div>
+          </div>
+        </div>
+        <div class="book-actions">
+          <a href="/api/epubs/${encodeURIComponent(f.name)}" download class="btn-book-action btn-book-download">Download</a>
+          <button type="button" onclick="triggerFix('${encodeURIComponent(f.name)}')" class="btn-book-action btn-book-fix">Fix</button>
+          <div class="delete-wrap" id="delWrap_${CSS.escape(f.name)}">
+            <button type="button" onclick="confirmDelete('${encodeURIComponent(f.name)}')" class="btn-book-action btn-book-delete">Delete</button>
+          </div>
+        </div>
+      `;
+      groupCard.appendChild(row);
+    }
+
+    if (g.fixed) {
+      const f = g.fixed;
+      const cleanTitle = f.name.replace(/\.epub$/i, '').replace(/[-_]/g, ' ');
+      const row = document.createElement('div');
+      row.className = 'book-edition-row fixed-edition';
+
+      row.innerHTML = `
+        <div class="book-info">
+          <svg class="book-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+            <path d="M4 19.5v-15A2.5 2.5 0 0 1 6.5 2H20v20H6.5a2.5 2.5 0 0 1-2.5-2.5Z"/>
+            <path d="M6 6h10"/>
+            <path d="M6 10h10"/>
+          </svg>
+          <div class="book-meta-block">
+            <div class="book-title-line">
+              <span class="book-title" title="${cleanTitle}">${cleanTitle}</span>
+              <span class="badge-fixed">Polished Edition</span>
+            </div>
+            <div class="book-specs">${f.size_human} • ${new Date(f.mtime * 1000).toLocaleDateString()} ${new Date(f.mtime * 1000).toLocaleTimeString([], {hour: '2-digit', minute: '2-digit'})}</div>
+          </div>
+        </div>
+        <div class="book-actions">
+          <a href="/api/epubs/${encodeURIComponent(f.name)}" download class="btn-book-action btn-book-download">Download</a>
+          <div class="delete-wrap" id="delWrap_${CSS.escape(f.name)}">
+            <button type="button" onclick="confirmDelete('${encodeURIComponent(f.name)}')" class="btn-book-action btn-book-delete">Delete</button>
+          </div>
+        </div>
+      `;
+      groupCard.appendChild(row);
+    }
+
+    container.appendChild(groupCard);
+  }
+}
+
+function confirmDelete(encodedName) {
+  const name = decodeURIComponent(encodedName);
+  const wrap = document.getElementById(`delWrap_${CSS.escape(name)}`);
+  if (!wrap) return;
+  wrap.innerHTML = `
+    <div class="delete-confirm-box">
+      <button type="button" onclick="executeDelete('${encodeURIComponent(name)}')" class="btn-confirm-delete">Confirm</button>
+      <button type="button" onclick="loadEpubs()" class="btn-cancel-delete">Cancel</button>
+    </div>
+  `;
+}
+
+document.addEventListener('keydown', (e) => {
+  if (e.key === 'Escape') {
+    const confirmBoxes = document.querySelectorAll('.delete-confirm-box');
+    if (confirmBoxes.length > 0) {
+      loadEpubs();
+    }
+  }
+});
+
+async function executeDelete(encodedName) {
+  try {
+    const res = await fetch(`/api/epubs/${encodedName}`, { method: 'DELETE' });
+    if (!res.ok) throw new Error('Delete failed');
+    showToast('EPUB removed from library', 'info');
+    loadEpubs();
+  } catch(e) {
+    showToast('Failed to delete file', 'error');
+  }
+}
+
+async function syncActiveJob() {
+  try {
+    const res = await fetch('/api/jobs/active');
+    if (res.ok) {
+      const data = await res.json();
+      if (data.active && data.job) {
+        currentJobId = data.job.id;
+        localStorage.setItem('silksoup_active_job', currentJobId);
+        showActive(data.job);
+        connectSSE(currentJobId);
+        return;
+      }
+    }
+  } catch(e) {}
+
+  if (currentJobId) {
+    try {
+      const res = await fetch(`/api/jobs/${currentJobId}`);
+      if (res.ok) {
+        const j = await res.json();
+        if (j.status === 'running' || j.status === 'queued') {
+          showActive(j);
+          connectSSE(currentJobId);
+        } else {
+          showActive(j);
+          refreshJob();
+          setFormLocked(false);
+        }
+      } else {
+        localStorage.removeItem('silksoup_active_job');
+        setFormLocked(false);
+      }
+    } catch(e) {
+      localStorage.removeItem('silksoup_active_job');
+      setFormLocked(false);
+    }
+  } else {
+    setFormLocked(false);
+  }
+}
+
+updateNotifyButton();
+syncActiveJob();
+loadEpubs();
+</script>
+</body>
+</html>