From 3927d11bffb78e555435e2e34bbf84f281494439 Mon Sep 17 00:00:00 2001 From: PouyaKhajavi Date: Sat, 8 Aug 2026 11:50:04 +0330 Subject: [PATCH] docs: add checklist.md to track project progress --- checklist.md | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 checklist.md diff --git a/checklist.md b/checklist.md new file mode 100644 index 0000000..d417af1 --- /dev/null +++ b/checklist.md @@ -0,0 +1,44 @@ +# Project Checklist - Intelligent Divar Ads Crawler + +This checklist tracks the implementation progress of the Intelligent Divar Ads Crawler with AI flagging and Telegram alerts. + +## 1. Project Setup & Infrastructure +- [x] Create Python virtual environment and set up `requirements.txt` +- [ ] Configure Docker environment (`docker-compose.yml` & `Dockerfiles`) +- [ ] Setup environment variables template (`.env.example`) + +## 2. Backend Base & Models +- [x] Implement core abstract base models (`BaseModel`, `TimeStampedModel` in `core/models.py`) +- [x] Implement Crawler app models (`CrawlTask` and `CrawlRun` in `crawler/models.py`) +- [x] Implement Ads app models (`Ad`, `AdEvaluation`, and `NotificationLog` in `ads/models.py`) +- [x] Generate database migrations and apply them + +## 3. API Views, Serializers, and Routing (DRF) +- [x] Create API health check endpoint (`/api/health/`) +- [x] Create `CrawlTask` and `CrawlRun` serializers and views +- [x] Implement manual trigger action (`/api/crawlers//trigger/`) and runs history (`/api/crawlers//runs/`) +- [x] Create `Ad`, `AdEvaluation`, and `NotificationLog` serializers and views +- [x] Implement ads filtering by `crawl_task` and `is_flagged` (`/api/ads/`) +- [x] Write backend unit tests for all implemented API endpoints and ensure they pass + +## 4. Background Tasks & Scheduler (Celery) +- [ ] Configure Celery app and connect to Django settings (`core/celery.py`) +- [ ] Setup Celery dynamic database scheduler (`django-celery-beat`) +- [ ] Implement crawler pipeline task (`run_crawl_pipeline` in `crawler/tasks.py`) +- [ ] Add scraping/extraction logic for Divar public listings + +## 5. AI Flagging & Telegram Notification +- [ ] Implement AI analysis with LLM Structured Outputs in `ads/tasks.py` +- [ ] Implement Telegram Bot notification sender in `ads/tasks.py` +- [ ] Connect full pipeline (Scrape -> AI Flag -> Telegram Alert) + +## 6. Frontend Dashboard (React + Vite) +- [ ] Initialize React project in `frontend/` +- [ ] Create layout and custom CSS styling +- [ ] Build Crawler Management page (list, create, edit, trigger, log view) +- [ ] Build Ads Feed page (display filtered ads with AI evaluation details) +- [ ] Connect React frontend to Django REST APIs + +## 7. Verification & Production Readiness +- [ ] Run full system manual QA (Edge cases, errors, network failure) +- [ ] Optimize logging, performance, and rate limiting