You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Mohsen Taba a28563e9a8 hadis sync N+1 fixed 5 months ago
..
README.md Refactor API Documentation System and optimize Hadis data scripts 10 months ago
__init__.py Add management commands for seeding basic and comprehensive Hadis data 11 months ago
fix_sects.py Add management commands for seeding basic and comprehensive Hadis data 11 months ago
seed_basic_data.py Add management commands for seeding basic and comprehensive Hadis data 11 months ago
seed_books.py hadis sync N+1 fixed 5 months ago
seed_category_data.py hadis sync N+1 fixed 5 months ago
seed_hadis_2.py hadis sync N+1 fixed 5 months ago
seed_hadis_data.py Refactor API Documentation System and optimize Hadis data scripts 10 months ago
seed_transmitters.py hadis sync N+1 fixed 5 months ago
test_safe_seeding.py Refactor API Documentation System and optimize Hadis data scripts 10 months ago
test_sects.py Add management commands for seeding basic and comprehensive Hadis data 11 months ago

README.md

Hadis Management Commands

seed_hadis_data

<<<<<<< HEAD This management command seeds comprehensive data for all Hadis app models with realistic sample records while maintaining proper relationships and business domain logic.

This management command seeds comprehensive data for all Hadis app models with realistic sample records while maintaining proper relationships and business domain logic. Enhanced with lock detection and retry logic to prevent database locks.

932fb17 (Refactor API Documentation System and optimize Hadis data scripts)

Usage

# Basic usage - seed data with default settings
python manage.py seed_hadis_data

# Clear existing data before seeding
python manage.py seed_hadis_data --clear

# Specify custom images directory
python manage.py seed_hadis_data --images-dir /path/to/images

# Specify custom XMind file
python manage.py seed_hadis_data --xmind-file /path/to/file.xmind

# Combine options
python manage.py seed_hadis_data --clear --images-dir scripts/seed_images --xmind-file scripts/test.xmind

Options

  • --clear: Clear existing hadis data before seeding (optional)
  • --images-dir: Directory containing seed images (default: scripts/seed_images)
  • --xmind-file: Path to XMind file for categories (default: scripts/test.xmind)

What it creates

  1. HadisStatus records: Various hadis authenticity statuses (Достоверный, Хороший, etc.)
  2. HadisTag records: Topic tags for categorizing hadis
  3. HadisSect records: Shia and Sunni sects
  4. HadisCategory records: Hierarchical categories for both Quran and Hadith sources
  5. Library data: Books, categories, and collections for references
  6. Transmitters: Historical figures who transmitted hadis
  7. Hadis records: Complete hadis with translations, explanations, and relationships
  8. Transmission chains: Links between hadis and transmitters
  9. References: Book references with images

Requirements

  • The images directory must contain PNG files for book covers and reference images
  • The XMind file is optional but recommended for category mind maps
  • All models must be properly migrated before running

<<<<<<< HEAD

Performance

The command uses optimized batch operations to create data efficiently:

  • Bulk create/update operations for categories
  • Checks for existing records to avoid duplicates
  • Progress reporting for large datasets =======

Performance & Lock Prevention

The command uses advanced techniques to prevent database locks and ensure reliable execution:

  • Lock Detection: Automatically detects database locks and deadlocks
  • Retry Logic: Retries failed operations with exponential backoff (up to 5 attempts)
  • Step-by-step Processing: Creates records individually with small delays to prevent locks
  • Batch Processing: Processes tags in small batches to avoid overwhelming the database
  • No Large Transactions: Avoids wrapping everything in atomic transactions that can cause locks
  • Progress Reporting: Detailed progress with emoji indicators and clear status messages
  • Error Handling: Graceful handling of duplicate records and constraint violations

932fb17 (Refactor API Documentation System and optimize Hadis data scripts)

Example Output

Starting Hadis data seeding...
Found 4 seed images
XMind file: scripts/test.xmind
Creating Hadis Statuses...
  Created status: Достоверный
  Created status: Хороший
...
Creating Hadis Categories...
  Creating categories for Шииты-двунадесятники...
    Batch created 6 Quran categories
...
Successfully seeded all Hadis data!

<<<<<<< HEAD

test_safe_seeding

A simple test command to verify that the lock detection and retry logic is working properly.

Usage

# Test the safe seeding functionality
python manage.py test_safe_seeding

What it tests

  • Database connectivity
  • Lock detection mechanisms
  • Retry logic for failed operations
  • Creation of test records (sect, status, tag)

Additional Commands

fix_sects

Fixes any issues with sect creation by using simple English titles.

python manage.py fix_sects

seed_basic_data

Creates only the essential basic data (statuses, tags, sects) without the full dataset.

python manage.py seed_basic_data [--clear]

932fb17 (Refactor API Documentation System and optimize Hadis data scripts)