dovodi
/
Dovodi_Backend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

making backup values for multilanguage fields, md file made for egress architecture

master
Mohsen Taba 1 month ago
parent
4019594d57
commit
da0a084c26
3 changed files with 454 additions and 7 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 8
      apps/blog/models.py
  2. 29
      apps/course/models/course.py
  3. 424
      docs/web_egress_architecture.md

8
apps/blog/models.py
View File

@ -106,11 +106,9 @@ class Blog(models.Model):
def get_blog_filed(self, lang, blog_field):
try:
if isinstance(blog_field, list) and blog_field:
for tr in blog_field:
if isinstance(tr, dict) and tr.get('language_code') == lang:
return tr.get('title') or tr.get('text') or tr.get('value')
return None
from apps.course.models.course import get_localized_field
return get_localized_field(lang, blog_field)
except Exception as exp:
print(f'---> Error in get_blog_filed: {exp}')
return None

29
apps/course/models/course.py
View File

@ -65,9 +65,34 @@ def format_multilingual_field(value):
def get_localized_field(lang, field_value):
try:
if isinstance(field_value, list) and field_value:
requested_lang = str(lang or "").lower()
fallback_map = {}
for tr in field_value:
if isinstance(tr, dict) and tr.get('language_code') == lang:
return tr.get('title') or tr.get('text') or tr.get('value') or tr.get('name')
if not isinstance(tr, dict):
continue
text = tr.get('title') or tr.get('text') or tr.get('value') or tr.get('name')
language_code = str(tr.get('language_code') or "").lower()
if language_code:
fallback_map[language_code] = text
if language_code == requested_lang and text:
return text
if fallback_map.get('en'):
return fallback_map['en']
if fallback_map.get('ru'):
return fallback_map['ru']
for tr in reversed(field_value):
if isinstance(tr, dict):
text = tr.get('title') or tr.get('text') or tr.get('value') or tr.get('name')
if text:
return text
return extract_text_from_json(field_value)
return extract_text_from_json(field_value)
except Exception as exp:

424
docs/web_egress_architecture.md
View File

@ -0,0 +1,424 @@
# Web Egress Recording Architecture
This document describes how online-class recording works after replacing the old PlugNMeet recorder with LiveKit Web Egress.
## Goal
We no longer rely on the built-in PlugNMeet recorder for class recordings.
Instead:
- PlugNMeet is still the meeting server and source of classroom events
- LiveKit Egress is responsible for recording the rendered meeting page
- Django remains the source of truth for sessions, recordings, lessons, and attendance-related data
- The chat/FastAPI service acts as the bridge between Django and LiveKit Egress
## Main Services
### 1. `conference_client`
Path:
- `Online Class/conference_client/`
Role:
- Renders the online classroom UI
- Provides the `Rec` button for professor recording control
- Hosts the same meeting page used by normal users and the recorder bot
- Applies recorder-specific UI rules when the access token belongs to the recorder
Important points:
- The recorder joins this same frontend using a special access token
- Some UI branches are customized for `isRecorder` / `web_egress=1`
- The recorder should bypass landing/prejoin steps and go directly into the meeting
### 2. `conference_server` / PlugNMeet
Path:
- `conference_server/conference_server/`
Role:
- Creates and runs meeting rooms
- Issues meeting join tokens
- Sends meeting lifecycle webhooks
- Continues to provide room state, participant lifecycle, and room end events
Important points:
- PlugNMeet still owns the real classroom
- Attendance and automatic session closing logic still depend on PlugNMeet room/webhook behavior
- Only the recording backend changed, not the meeting backend itself
### 3. Django backend
Path:
- `backend/apps/course/`
Role:
- Manages `CourseLiveSession`
- Generates join URLs and meeting access tokens
- Tracks `LiveSessionUser`
- Stores `LiveSessionRecording`
- Creates course lessons automatically from saved recordings
- Controls recording start/stop through backend APIs
Important files:
- [live_session.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/views/live_session.py)
- [course.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/views/course.py)
- [webhook.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/views/webhook.py)
- [signals.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/signals.py)
- [web_egress.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/services/web_egress.py)
### 4. Chat / FastAPI bridge service
Path:
- `chat/chat/`
Role:
- Receives recording start/stop requests from Django
- Talks to LiveKit Egress
- Tracks the current egress job state
- Receives LiveKit webhook callbacks
- Uploads the final recorded file back to Django
Important files:
- [web_egress_apis.py](/F:/WORK/CODE/WORK/imam-javad/chat/chat/web_egress_apis.py)
- [main.py](/F:/WORK/CODE/WORK/imam-javad/chat/chat/main.py)
- [settings.py](/F:/WORK/CODE/WORK/imam-javad/chat/chat/settings.py)
### 5. LiveKit + Egress stack
External/runtime project on server:
- LiveKit server
- Redis
- Caddy/L4 proxy
- LiveKit Egress
Role:
- Accepts recording jobs
- Opens the target classroom URL in a browser-like environment
- Captures the rendered meeting page
- Saves the output file to shared storage
- Emits webhook events back to the chat/FastAPI bridge
## High-Level Flow
```mermaid
flowchart LR
A["Professor clicks Rec in conference_client"] --> B["Django recording API"]
B --> C["WebEgressClient in Django"]
C --> D["Chat/FastAPI /api/web-egress/start/"]
D --> E["LiveKit Egress job created"]
E --> F["Recorder bot opens conference_client join URL"]
F --> G["Rendered meeting page is recorded"]
G --> H["Egress stores output file"]
H --> I["LiveKit webhook -> chat/FastAPI"]
I --> J["chat/FastAPI uploads file to Django callback"]
J --> K["Django creates LiveSessionRecording"]
K --> L["Django signal creates CourseLesson"]
```
## Recording Start Flow
### Frontend
Professor presses the recording button in:
- [useCloudRecording.tsx](/F:/WORK/CODE/WORK/imam-javad/Online%20Class/conference_client/src/components/footer/icons/recording/useCloudRecording.tsx)
That calls:
- `POST /api/courses/online/room/recording/`
Payload:
```json
{
"action": "start",
"room_id": "room-16-1781432247"
}
```
### Django
Handled in:
- `CourseLiveSessionRecordingAPIView.post()`
What Django does:
1. Resolves the current live session from the meeting token
2. Checks that the caller is a moderator
3. Marks session status as `starting`
4. Sends a start request to the Web Egress bridge service
5. Stores returned `egress_id`
6. Marks the session as `recording`
Important session fields used:
- `web_egress_id`
- `web_egress_status`
- `web_egress_started_at`
- `web_egress_stopped_at`
- `web_egress_error`
- `recording_title`
### Chat / FastAPI bridge
Handled in:
- `POST /api/web-egress/start/`
What it does:
1. Registers a job for the given `session_id` and `room_id`
2. Builds the join URL for the recorder bot
3. Calls LiveKit Egress to start a browser-based recording
4. Stores current job state
### Recorder Bot
The recorder joins the classroom using a special access token and URL generated by Django:
- `?access_token=...&web_egress=1&skip_landing=1&room_id=...`
This is important because:
- the recorder uses the real classroom UI
- it must bypass landing/prejoin checks
- it must render the same content users see
## Recording Stop Flow
### Manual stop
Professor presses `Rec` again:
- frontend calls `POST /api/courses/online/room/recording/` with `"action": "stop"`
Django:
1. verifies the session
2. sends stop request to Web Egress bridge
3. marks session as `stopping`
### Automatic stop on session end
This is critical.
If the professor ends the meeting while recording is still active, Django must still stop Web Egress before the session is fully closed.
This is now handled centrally by:
- `stop_session_web_egress_if_active(...)`
Used from:
- [course.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/views/course.py)
- [webhook.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/views/webhook.py)
- [admin.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/views/admin.py)
- [live_session.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/views/live_session.py)
This ensures the final active recording is not lost when the class is closed.
## Finalization Flow
After stop:
1. LiveKit Egress finishes processing the output file
2. LiveKit sends webhook events to the chat/FastAPI bridge
3. The bridge locates the output file in shared storage
4. The bridge uploads the file to Django callback endpoint
5. Django creates `LiveSessionRecording`
6. Django updates `CourseLiveSession.web_egress_status`
Important Django callback endpoint:
- `POST /api/courses/online/room/recording/web-egress/callback/`
Handled in:
- `CourseLiveSessionWebEgressCallbackAPIView`
## Shared Storage
The recording file is not sent directly from LiveKit to Django first.
Current pattern:
1. Egress writes file to shared disk
2. Chat/FastAPI reads the file
3. Chat/FastAPI uploads file to Django
Example shared path on server:
- `/najm/HabibMeetSocket/shared/recordings/imamjavad/web-egress/<room-id>/`
This means:
- LiveKit Egress and chat/FastAPI must both be able to access the same recording storage
- file permissions and mount paths are important
## Session and Recording Data Model
### `CourseLiveSession`
Represents the live class session.
Important responsibilities:
- room identity
- live/ended status
- moderator auto-close timing
- current Web Egress recording state
### `LiveSessionUser`
Represents user presence inside a live session.
Used for:
- attendance-like tracking
- determining who is online
- moderator-exit auto-close logic
Important note:
Even with webhooks, Django also performs fail-safe registration when direct join-token flows are used, so presence data is not lost when some event paths are bypassed.
### `LiveSessionRecording`
Represents a final recorded file stored in Django.
Used for:
- listing recordings
- generating lessons
- attaching final media files to courses
## Lesson Creation Flow
When a `LiveSessionRecording` is created, Django signal logic automatically creates a lesson.
Handled in:
- [signals.py](/F:/WORK/CODE/WORK/imam-javad/backend/apps/course/signals.py)
What it does:
1. Creates or reuses a chapter for the recording date
2. Creates `Lesson`
3. Creates `CourseLesson`
4. Uses session title or subject as lesson title
5. Adds `part 1`, `part 2`, etc. when multiple recordings belong to the same session
### Late title update behavior
If professor enters a custom recording title near session end, old recordings and already-created lessons must also be renamed.
This is now handled by syncing titles after `recording_title` is updated, so early-created lessons do not keep stale names.
## Why We Kept PlugNMeet Events
The recorder changed, but PlugNMeet room lifecycle still matters.
Things that should still come from PlugNMeet or session logic around it:
- room end state
- participant join/leave tracking
- presence rows in `LiveSessionUser`
- moderator exit detection
- automatic session closure
So the architecture is intentionally split:
- PlugNMeet remains the meeting authority
- Web Egress becomes only the recording engine
- Django links both worlds together
## Important Endpoints
### Django
- `POST /api/courses/online/room/recording/`
- start or stop recording
- `GET /api/courses/online/room/recording/?room_id=...`
- read recording status
- `POST /api/courses/online/room/recording/web-egress/callback/`
- receive final uploaded recording from bridge service
- `POST /api/courses/online/room/set-recording-title/`
- set custom recording title for active session
- `POST /api/courses/plugnmeet/webhook/`
- receive room and participant lifecycle events from PlugNMeet
### Chat / FastAPI bridge
- `POST /api/web-egress/start/`
- `POST /api/web-egress/stop/`
- `GET /api/web-egress/status/`
- `POST /api/web-egress/livekit-webhook/`
- `GET /api/web-egress/health/`
## Environment and Runtime Dependencies
### Django
Relevant envs:
- `ONLINE_CLASS_WEB_EGRESS_ENABLED`
- `ONLINE_CLASS_WEB_EGRESS_SERVICE_URL`
- `ONLINE_CLASS_WEB_EGRESS_SERVICE_TOKEN`
- `ONLINE_CLASS_WEB_EGRESS_CALLBACK_TOKEN`
- `ONLINE_CLASS_WEB_EGRESS_TIMEOUT`
### Chat / FastAPI bridge
Must know:
- Django upload/callback target
- callback token
- shared recordings root
- LiveKit/Egress credentials
### LiveKit/Egress
Must be configured with:
- correct output path
- webhook target to chat/FastAPI
- access to the shared recordings directory
## Current Operational Notes
- The recorder is a hidden meeting participant named `Session Recorder`
- The recorder uses the real meeting frontend, not a separate rendering app
- The frontend has recorder-specific behavior to skip entry/landing flow
- Presence and auto-close logic must continue to work independently of recording
- Recording stop should be triggered both manually and during session close
## Summary
The current architecture is a three-layer recording pipeline:
1. PlugNMeet hosts the class
2. Django decides when recording starts/stops and stores final course data
3. Chat/FastAPI bridges Django to LiveKit Egress
LiveKit Egress records the rendered classroom page, but PlugNMeet remains the source of meeting truth.
That split is the key design decision:
- do not move classroom lifecycle into Egress
- only move the recording responsibility into Egress
Write Preview
Loading…
Cancel
Save