# Troubleshooting Import Issues

## Encoding Errors

### Problem: "charmap codec can't decode byte"
**Symptoms:**
```
Fatal error: 'charmap' codec can't decode byte 0x9d in position 7244: character maps to <undefined>
```

**Solution:**
This has been fixed in the latest version. If you still see this error:

1. **Rebuild Docker container** (changes don't apply until rebuild):
   ```bash
   docker-compose down
   docker-compose build
   docker-compose up -d
   ```

2. **Verify the fix is active**:
   ```bash
   # Check encoding order includes iso-8859-1 early
   docker-compose exec delphi-db grep "encodings = " app/import_legacy.py
   
   # Should show: ["utf-8", "utf-8-sig", "iso-8859-1", "latin-1", ...]
   ```

3. **Check container is using new image**:
   ```bash
   docker-compose ps
   # Note the CREATED time - should be recent
   ```

### Problem: File not found during import

**Symptoms:**
- "File not found" error
- Import appears to succeed but no data imported

**Solution:**
1. Make sure files are in the `data-import/` directory
2. Files must be accessible inside the Docker container
3. Check docker-compose.yml mounts the correct volume:
   ```yaml
   volumes:
     - ./data-import:/app/data-import
   ```

## Import Workflow Issues

### Problem: Import hangs or takes too long

**Symptoms:**
- Import never completes
- Browser shows loading indefinitely

**Solution:**
1. Check Docker logs:
   ```bash
   docker-compose logs -f delphi-db
   ```

2. Large files (50k+ rows) may take several minutes
3. Check for duplicate key violations in logs

### Problem: "Unknown import type"

**Symptoms:**
- File uploads but shows as "unknown"
- Cannot auto-import

**Solution:**
1. File must match expected naming pattern (e.g., `rolodex_*.csv`)
2. Use manual mapping in admin interface:
   - Go to Admin → Import section
   - Find the unknown file
   - Click "Map to Import Type"
   - Select correct import type
   - Save and import

## Database Issues

### Problem: Duplicate key violations

**Symptoms:**
```
UNIQUE constraint failed: rolodex.id
IntegrityError: duplicate key value
```

**Solution:**
1. Data already exists - safe to ignore if re-importing
2. To clear and re-import:
   - Delete existing database: `rm delphi.db`
   - Restart container: `docker-compose restart`
   - Import files in correct order (reference data first)

### Problem: Foreign key violations

**Symptoms:**
```
FOREIGN KEY constraint failed
```

**Solution:**
Import files in dependency order:
1. Reference tables first (employee, filetype, etc.)
2. Core tables next (rolodex, files)
3. Related tables last (phone, ledger, payments)

See `IMPORT_ORDER` in admin interface for correct sequence.

## Debugging Steps

### 1. Check Application Logs
```bash
docker-compose logs -f delphi-db
```

### 2. Access Container Shell
```bash
docker-compose exec delphi-db bash
```

### 3. Test Import Manually
```python
# Inside container
python3 << EOF
from app.import_legacy import open_text_with_fallbacks
import csv

f, encoding = open_text_with_fallbacks('data-import/rolodex_*.csv')
print(f"Encoding: {encoding}")
reader = csv.DictReader(f)
print(f"Columns: {reader.fieldnames}")
EOF
```

### 4. Check File Permissions
```bash
ls -la data-import/
# Files should be readable
```

### 5. Verify Python Environment
```bash
docker-compose exec delphi-db python3 --version
docker-compose exec delphi-db pip list | grep -i sql
```

## Common Mistakes

1. **Not rebuilding after code changes** - Docker containers use cached images
2. **Wrong file format** - Must be CSV with headers
3. **File encoding not supported** - Should be handled by fallback, but exotic encodings may fail
4. **Importing in wrong order** - Dependencies must be imported first
5. **Missing required columns** - Check CSV has expected columns

## Getting Help

If issues persist:
1. Check logs: `docker-compose logs delphi-db`
2. Note exact error message and stack trace
3. Check which import function is failing
4. Verify file format matches expected schema
5. Try importing a small test file (10 rows) to isolate issue