Commit Graph

10 Commits

Author SHA1 Message Date
HotSwapp
63809d46fb Fix PHONE.csv import duplicate constraint error
- Implement upsert logic in import_phone() function
- Check for existing (id, phone) combinations before insert
- Track duplicates within CSV to skip gracefully
- Update existing records instead of failing on duplicates
- Add detailed statistics: inserted, updated, skipped counts
- Align with upsert pattern used in other import functions
- Add documentation in docs/PHONE_IMPORT_FIX.md

Fixes: UNIQUE constraint failed: phone.id, phone.phone error
when re-importing or uploading CSV with duplicate entries
2025-10-12 21:45:30 -05:00
HotSwapp
22e99d27ed Fix UNIQUE constraint errors in reference table imports with upsert logic
- Implement upsert (INSERT or UPDATE) logic for all reference table imports
- Fixed functions: import_trnstype, import_trnslkup, import_footers,
  import_filestat, import_employee, import_gruplkup, import_filetype,
  import_fvarlkup, import_rvarlkup
- Now checks if record exists before inserting; updates if exists
- Makes imports idempotent - can safely re-run without errors
- Added tracking of inserted vs updated counts in result dict
- Maintains batch commit performance for large imports
- Fixes sqlite3.IntegrityError when re-importing CSV files
2025-10-12 21:36:28 -05:00
HotSwapp
ad1c75d759 docs: Add comprehensive guide on duplicate record handling 2025-10-12 21:08:38 -05:00
HotSwapp
789eb2c134 docs: Add comprehensive troubleshooting guide for import issues 2025-10-12 20:13:55 -05:00
HotSwapp
89ff90a384 docs: Add comprehensive documentation of CSV encoding fix 2025-10-12 19:19:56 -05:00
HotSwapp
7958556613 Fix: Improved CSV encoding detection for legacy data with non-standard characters
- Changed encoding fallback order to prioritize iso-8859-1/latin-1 over cp1252
- Increased encoding test from 1KB to 10KB to catch issues deeper in files
- Added proper file handle cleanup on encoding failures
- Resolves 'charmap codec can't decode byte 0x9d' error in rolodex import
- Tested with rolodex file containing 52,100 rows successfully
2025-10-12 19:19:25 -05:00
HotSwapp
2e7e9693c5 Add next section prompt and task summary 2025-10-08 09:55:12 -05:00
HotSwapp
e11e9aaf16 Add comprehensive CSV import system documentation
- Created IMPORT_GUIDE.md: Complete user guide with step-by-step instructions
  * Detailed import order for all 27+ tables
  * Troubleshooting guide
  * Data validation procedures
  * Best practices and performance notes

- Created IMPORT_SYSTEM_SUMMARY.md: Technical implementation summary
  * Complete list of all implemented functions (28 import + 7 sync)
  * Architecture and data flow diagrams
  * Module organization
  * Testing status and next steps
  * ~3,000 lines of code documented
2025-10-08 09:54:30 -05:00
HotSwapp
684b947651 docs: add next-section prompt for Reports (Envelope, Phone Book variants, Rolodex Info); confirm TODO next step pending run/test 2025-10-07 17:40:02 -05:00
HotSwapp
432f303a33 docs: add inferred legacy schema from CSV headers and .SC usage to guide migration 2025-10-07 09:51:30 -05:00