From 789eb2c134c810a2b4fbd53a504f0371e63c636f Mon Sep 17 00:00:00 2001
From: HotSwapp <47397945+HotSwapp@users.noreply.github.com>
Date: Sun, 12 Oct 2025 20:13:55 -0500
Subject: [PATCH] docs: Add comprehensive troubleshooting guide for import
 issues

---
 docs/TROUBLESHOOTING_IMPORTS.md | 168 ++++++++++++++++++++++++++++++++
 1 file changed, 168 insertions(+)
 create mode 100644 docs/TROUBLESHOOTING_IMPORTS.md

diff --git a/docs/TROUBLESHOOTING_IMPORTS.md b/docs/TROUBLESHOOTING_IMPORTS.md
new file mode 100644
index 0000000..5c3cf00
--- /dev/null
+++ b/docs/TROUBLESHOOTING_IMPORTS.md
@@ -0,0 +1,168 @@
+# Troubleshooting Import Issues
+
+## Encoding Errors
+
+### Problem: "charmap codec can't decode byte"
+**Symptoms:**
+```
+Fatal error: 'charmap' codec can't decode byte 0x9d in position 7244: character maps to <undefined>
+```
+
+**Solution:**
+This has been fixed in the latest version. If you still see this error:
+
+1. **Rebuild Docker container** (changes don't apply until rebuild):
+   ```bash
+   docker-compose down
+   docker-compose build
+   docker-compose up -d
+   ```
+
+2. **Verify the fix is active**:
+   ```bash
+   # Check encoding order includes iso-8859-1 early
+   docker-compose exec delphi-db grep "encodings = " app/import_legacy.py
+   
+   # Should show: ["utf-8", "utf-8-sig", "iso-8859-1", "latin-1", ...]
+   ```
+
+3. **Check container is using new image**:
+   ```bash
+   docker-compose ps
+   # Note the CREATED time - should be recent
+   ```
+
+### Problem: File not found during import
+
+**Symptoms:**
+- "File not found" error
+- Import appears to succeed but no data imported
+
+**Solution:**
+1. Make sure files are in the `data-import/` directory
+2. Files must be accessible inside the Docker container
+3. Check docker-compose.yml mounts the correct volume:
+   ```yaml
+   volumes:
+     - ./data-import:/app/data-import
+   ```
+
+## Import Workflow Issues
+
+### Problem: Import hangs or takes too long
+
+**Symptoms:**
+- Import never completes
+- Browser shows loading indefinitely
+
+**Solution:**
+1. Check Docker logs:
+   ```bash
+   docker-compose logs -f delphi-db
+   ```
+
+2. Large files (50k+ rows) may take several minutes
+3. Check for duplicate key violations in logs
+
+### Problem: "Unknown import type"
+
+**Symptoms:**
+- File uploads but shows as "unknown"
+- Cannot auto-import
+
+**Solution:**
+1. File must match expected naming pattern (e.g., `rolodex_*.csv`)
+2. Use manual mapping in admin interface:
+   - Go to Admin → Import section
+   - Find the unknown file
+   - Click "Map to Import Type"
+   - Select correct import type
+   - Save and import
+
+## Database Issues
+
+### Problem: Duplicate key violations
+
+**Symptoms:**
+```
+UNIQUE constraint failed: rolodex.id
+IntegrityError: duplicate key value
+```
+
+**Solution:**
+1. Data already exists - safe to ignore if re-importing
+2. To clear and re-import:
+   - Delete existing database: `rm delphi.db`
+   - Restart container: `docker-compose restart`
+   - Import files in correct order (reference data first)
+
+### Problem: Foreign key violations
+
+**Symptoms:**
+```
+FOREIGN KEY constraint failed
+```
+
+**Solution:**
+Import files in dependency order:
+1. Reference tables first (employee, filetype, etc.)
+2. Core tables next (rolodex, files)
+3. Related tables last (phone, ledger, payments)
+
+See `IMPORT_ORDER` in admin interface for correct sequence.
+
+## Debugging Steps
+
+### 1. Check Application Logs
+```bash
+docker-compose logs -f delphi-db
+```
+
+### 2. Access Container Shell
+```bash
+docker-compose exec delphi-db bash
+```
+
+### 3. Test Import Manually
+```python
+# Inside container
+python3 << EOF
+from app.import_legacy import open_text_with_fallbacks
+import csv
+
+f, encoding = open_text_with_fallbacks('data-import/rolodex_*.csv')
+print(f"Encoding: {encoding}")
+reader = csv.DictReader(f)
+print(f"Columns: {reader.fieldnames}")
+EOF
+```
+
+### 4. Check File Permissions
+```bash
+ls -la data-import/
+# Files should be readable
+```
+
+### 5. Verify Python Environment
+```bash
+docker-compose exec delphi-db python3 --version
+docker-compose exec delphi-db pip list | grep -i sql
+```
+
+## Common Mistakes
+
+1. **Not rebuilding after code changes** - Docker containers use cached images
+2. **Wrong file format** - Must be CSV with headers
+3. **File encoding not supported** - Should be handled by fallback, but exotic encodings may fail
+4. **Importing in wrong order** - Dependencies must be imported first
+5. **Missing required columns** - Check CSV has expected columns
+
+## Getting Help
+
+If issues persist:
+1. Check logs: `docker-compose logs delphi-db`
+2. Note exact error message and stack trace
+3. Check which import function is failing
+4. Verify file format matches expected schema
+5. Try importing a small test file (10 rows) to isolate issue
+