69f1043be3
- Enhanced get_import_type_from_filename() to recognize model class names (LegacyFile, FilesR, etc.) in addition to legacy CSV names - Added import functions for States, Printers, and Setup reference tables - Updated VALID_IMPORT_TYPES and IMPORT_ORDER to include new tables - Updated admin panel table counts to display new reference tables - Created UPLOAD_FIX.md documentation explaining the changes and how to handle existing unknown files This fixes the issue where files uploaded with model class names (e.g., LegacyFile.csv) were being categorized as 'unknown' instead of being properly detected.
104 lines
4.1 KiB
Markdown
104 lines
4.1 KiB
Markdown
# Upload Detection Fix Summary
|
|
|
|
## Problem
|
|
Files uploaded to the admin panel were being detected as "unknown" when using model class names instead of legacy CSV names.
|
|
|
|
## Solution Implemented
|
|
|
|
### 1. Enhanced Filename Detection
|
|
Updated `get_import_type_from_filename()` in `app/main.py` to recognize both:
|
|
- **Legacy CSV names**: `FILES.csv`, `LEDGER.csv`, `PAYMENTS.csv`
|
|
- **Model class names**: `LegacyFile.csv`, `Ledger.csv`, `LegacyPayment.csv`
|
|
|
|
### 2. Added Support for Additional Tables
|
|
Added import functions and detection for three previously unsupported tables:
|
|
- **States** (STATES.csv) - US state abbreviations
|
|
- **Printers** (PRINTERS.csv) - Printer configuration
|
|
- **Setup** (SETUP.csv) - Application configuration
|
|
|
|
These are reference tables that should be imported early in the process.
|
|
|
|
## Filename Variations Now Supported
|
|
|
|
### Core Data Tables
|
|
| Model Class | Supported Filenames | Import Type |
|
|
|------------|---------------------|-------------|
|
|
| LegacyFile | `FILES.csv`, `FILE.csv`, `LegacyFile.csv` | `files` |
|
|
| FilesR | `FILES_R.csv`, `FILESR.csv`, `FilesR.csv` | `files_r` |
|
|
| FilesV | `FILES_V.csv`, `FILESV.csv`, `FilesV.csv` | `files_v` |
|
|
| FileNots | `FILENOTS.csv`, `FILE_NOTS.csv`, `FileNots.csv` | `filenots` |
|
|
| Ledger | `LEDGER.csv`, `Ledger.csv` | `ledger` |
|
|
| LegacyPayment | `PAYMENTS.csv`, `PAYMENT.csv`, `LegacyPayment.csv` | `payments` |
|
|
| LegacyPhone | `PHONE.csv`, `LegacyPhone.csv` | `phone` |
|
|
|
|
### New Reference Tables
|
|
| Model Class | Supported Filenames | Import Type |
|
|
|------------|---------------------|-------------|
|
|
| States | `STATES.csv`, `States.csv` | `states` |
|
|
| Printers | `PRINTERS.csv`, `Printers.csv` | `printers` |
|
|
| Setup | `SETUP.csv`, `Setup.csv` | `setup` |
|
|
|
|
## For Existing Unknown Files
|
|
|
|
If you have files already uploaded as `unknown_*.csv`, you have two options:
|
|
|
|
### Option 1: Re-upload with Correct Names
|
|
1. Delete the unknown files from the admin panel
|
|
2. Re-upload with any of the supported filename variations above
|
|
3. Files will now be auto-detected correctly
|
|
|
|
### Option 2: Use the Map Functionality
|
|
1. In the admin panel, find the "Unknown Data" section
|
|
2. Select the unknown files you want to map
|
|
3. Choose the target import type from the dropdown (e.g., `files`, `ledger`, `payments`)
|
|
4. Click "Map Selected" to rename them with the correct prefix
|
|
5. Import them using the import button
|
|
|
|
## Checking Unknown Files
|
|
To identify what type an unknown file might be, you can check its header row:
|
|
|
|
```bash
|
|
head -1 data-import/unknown_*.csv
|
|
```
|
|
|
|
Common headers:
|
|
- **LEDGER**: `File_No,Date,Item_No,Empl_Num,T_Code,T_Type,T_Type_L,Quantity,Rate,Amount,Billed,Note`
|
|
- **STATES**: `Abrev,St`
|
|
- **PRINTERS**: `Number,Name,Port,Page_Break,Setup_St,...`
|
|
- **SETUP**: `Appl_Title,L_Head1,L_Head2,L_Head3,...`
|
|
|
|
## Note on TRNSACTN Files
|
|
If you see unknown files with headers like:
|
|
```
|
|
File_No,Id,Footer_Code,Date,Item_No,Empl_Num,T_Code,T_Type,T_Type_L,Quantity,Rate,Amount,Billed,Note
|
|
```
|
|
|
|
These are **TRNSACTN** files (transaction join tables). TRNSACTN is a legacy reporting view that combines LEDGER with related tables. Currently, TRNSACTN import is not supported because it's a derived/joined view. The data should be imported via the individual tables (LEDGER, FILES, etc.) instead.
|
|
|
|
## Testing the Fix
|
|
|
|
1. Try uploading a file named `LegacyFile.csv` - should be detected as `files`
|
|
2. Try uploading `Ledger.csv` - should be detected as `ledger`
|
|
3. Try uploading `States.csv` - should be detected as `states`
|
|
4. Check the admin panel to see files grouped by their detected type (not "unknown")
|
|
5. Import as normal using the import buttons
|
|
|
|
## Changes Made
|
|
|
|
### Files Modified:
|
|
- `app/main.py`:
|
|
- Enhanced `get_import_type_from_filename()` with model class name detection
|
|
- Added `states`, `printers`, `setup` to `VALID_IMPORT_TYPES`
|
|
- Added new tables to `IMPORT_ORDER`
|
|
- Added import functions to `process_csv_import()`
|
|
- Updated `table_counts` in admin panel to show new tables
|
|
|
|
- `app/import_legacy.py`:
|
|
- Added `import_states()` function
|
|
- Added `import_printers()` function
|
|
- Added `import_setup()` function
|
|
- Imported States, Printers, Setup models
|
|
|
|
No database schema changes were needed - all three models already existed.
|
|
|