Think & Speak AI English Platform

Technical Architecture Workshop

A comprehensive deep-dive into the AI-powered English learning platform designed for Hong Kong classrooms. From phoneme-level pronunciation feedback to real-time multilingual translation—discover the technology that transforms how students learn English.

🎤 Pronunciation AI 📚 Reading Comprehension 🌐 Live Translation 📖 Vocabulary Builder
16kHz
Optimized Sample Rate
5
Practice Tasks
4
Assessment Dimensions
3
Languages Supported
1 / 12

Course Adventure: Learning Architecture

State Management & Data Flow
📚
Course Discovery Layer
Manages course catalog with real-time filtering and sorting. Default sort: "Most recently practiced." Pagination: 6 items per page for optimal cognitive load. The `refreshCourseList` boolean flag enables optimistic UI updates.
🧠
Session Tracking
Tracks learning position through `practicingCourseId` → `practicingLessonId` → `practicingTaskId` forming a nested navigation stack. Enables seamless session resumption across devices and page navigations.
💾
State Persistence
The `practicingTask` stores the active task object enabling instant state restoration on browser refresh. Supports progressive learning where students never lose their place.

Architecture Pattern

The Redux toolkit manages the complete learning journey through a nested state structure. `selectThinkAndSpeak` selector retrieves module-specific state. `thinkAndSpeakActions` dispatches curriculum updates.

// State Flow Example
const courseListData = useSelector(selectWords.listData);
const filtering = useSelector(selectWords.wordListFiltering);
dispatch(wordsAction.setWordListFiltering({ ...filter }));
// → Triggers curriculum re-render
Key Insight: The nested state structure means every action a student takes is captured and associated with their learning context. Teachers can see complete history of practice sessions and performance trends.
2 / 12

Phoneme-Level Pronunciation Feedback

AI-Powered Assessment

The Four-Dimensional Scoring Engine

When a student submits their recording, the AI breaks their speech into individual phonemes and compares each against native speaker models. This is not simple "right/wrong"—it identifies exactly which sounds need work.

Accuracy
Phonetic Precision
Fluency
Rhythm & Prosody
Completeness
Full Word Coverage
Prosody
Intonation Patterns

Score Color System

91–100: Excellent — native-like accuracy
71–90: Great — highly intelligible
61–70: Good Try — needs practice
51–60: Not Bad — significant errors
1–50: Needs Work — multiple errors
Differentiator: Most apps say "Correct" or "Incorrect." Ours shows exactly which phoneme was wrong and provides the correct pronunciation as a hint for targeted remediation.
3 / 12

5-Stage Vocabulary Acquisition Pipeline

Spaced Repetition & Mastery
🎙️
Pronunciation
35%
→
✏️
Spelling
35%
→
📝
Usage I
20%
→
📖
Usage II
20%
→
🖼️
Image Dictation
25%
🎯
Mastery Progression
Each task type maps to specific pedagogical outcomes. Spelling carries 35% weighting—orthographic precision indicates deep encoding critical for academic writing.
📊
Scoring Philosophy
Usage tasks carry 20% each—they test whether students can apply vocabulary in authentic contexts. Visual association supports rapid recall through multimodal encoding.
🎮
Gamified Retry System
Pronunciation: 5 attempts (physiologically complex). Usage: 3 attempts (controllable application). Each retry applies score penalties enforcing deliberate practice over brute-force.
Spaced Repetition: Words with lower avgScore appear more frequently. As scores improve, review intervals increase—forgetting curve optimization supported by cognitive science research.
4 / 12

AI Speech Assessment & Scoring Engine

Azure Cognitive Services
🤖
AI Processing Pipeline

The `utils.ts` file reveals the phoneme-level analysis engine. The code parses Azure Speech Service JSON responses containing phoneme-by-phoneme accuracy scores, filters for practices with valid assessment data (excluding recording errors or timeouts), and implements temporal sorting to identify the most recent practice attempt for adaptive feedback.

Phoneme-Level Analysis

// Sample PhonemeScore output
phonemeScores: [
  { phoneme: "/θ/", score: 85 },
  { phoneme: "/æ/", score: 72 },
  { phoneme: "/ɛ/", score: 91 }
]

Multi-Modal Content Architecture

The `VocabWordAudioContentType` definitions demonstrate a multi-modal content strategy:

  • ▸ VOCAB_WORD — Target word pronunciation
  • ▸ VOCAB_WORD_EXAMPLE — Contextualized pronunciation
  • ▸ VOCAB_WORD_EXPLANATION — Metalinguistic audio
  • ▸ VOCAB_WORD_SYNONYM — Semantic network building
Key Differentiator: This granular feedback transforms vague "right/wrong" into actionable "Your /θ/ sounds like /s/. Here's how to fix it." — enabling students to self-correct with precision.
5 / 12

Word List & Smart Filtering System

Vocabulary Management

Performance Optimization

The component handles potentially large word lists (1000+ items) through `useMemo` filtering with O(n) complexity. The `listDataFiltered` computation applies multiple simultaneous filter states including mastery status, practice recency, and alphabetical ordering.

Five-Dimensional Filtering

⏰ Time — 1W, 2W, 1M, 3M, 6M+
📊 Score — Ranges dynamically adjust by context
🔢 Practice Count — 0, 1-5, 6-10, 10+
🏷️ Category — Part of speech with Select All
🔍
Dynamic Score Ranges
For "Wrong Words" context: ranges below mastery threshold (0-50, 51-60, 61-70, 71-80, 81-90). For "Archived Words" context: ranges above mastery threshold (71-80, 81-90, 91-100). Prevents filtering to empty result sets.
🔗
Redux Integration
Each filter change triggers `handleFilterItemXXXApply()` → updates Redux state via `dispatch(wordsAction.setWordListFiltering(...))` → resets pagination → triggers word list refresh → updates URL query parameters.
Teacher Tip: Combine filters to create targeted vocabulary sets—"all verbs with scores below 60% not practiced in a month" for focused review sessions.
6 / 12

Mastery Classification & Progress

Spaced Repetition

Active vs. Mastered Vocabulary

The system implements a binary mastery classification: Active Learning Mode displays "Not Passed" words requiring continued spaced repetition practice. Mastery Library displays archived words exceeding the `WORD_TYPE_ARCHIVED_SCORE_MIN` threshold (80-90%) moved to "Your Library" for periodic maintenance.

Active Learning
Score ≤ 90% | Not Passed
Mastered
Score > 90% | Your Library

Progress Visualization

Total Words128
Mastered42
In Progress86
Cognitive Load: Separating mastered from active ensures students focus on what needs practice—maximum time on words that matter, minimal review of already-learned vocabulary.
7 / 12

Live Translation & Real-Time Infrastructure

WebSocket Architecture
🌐
Real-Time Communication
WebSocket-based real-time communication enables sub-100ms latency for live classroom sessions. Redis-backed session management stores transcription buffers, engagement signals, and teacher broadcasts in `CharRoom:{sessionId}`.
📡
Hybrid Encryption
The `HybridEncryptor` patterns secure Azure access tokens before transmission to student devices. Tokens are embedded in presigned URLs with configurable expiry. The `IP2Location` service enables O(log n) geolocation lookups.
Bilingual Support: English ↔ Mandarin ↔ Cantonese with intelligent auto-routing. English defaults to Cantonese translation for Hong Kong market alignment.

Learning Loop Pipeline

Listen — Real-time bilingual subtitles via WebSocket
Capture — Save sentences to My Notes with personal remarks
Review — Vocabulary extraction auto-generates flashcards
Master — Spaced repetition ensures long-term retention
Key Value: Every classroom utterance becomes a potential vocabulary card, practice script, or comprehension check. Every word is analyzed sound by sound.
8 / 12

Gamification & Progress Psychology

Engagement Architecture
🏆
Achievement System
Students earn badges for milestone completion, streak maintenance, and vocabulary mastery. XP accumulation creates visible progression that sustains long-term motivation.
🎯
Loss Aversion Design
Progress bars and streak counters leverage loss aversion—students don't want to break their streak or see progress bars drop, maintaining consistent daily practice.
💙
Mascot Companion
The blue anthropomorphic mascot provides emotional support during solo study. Non-judgmental design creates safe learning environment. Animated states (celebrating, encouraging) reduce anxiety.

Points Economy

XP
Experience Points
Badges
Achievement Milestones
Streaks
Daily Consistency
Levels
Course Adventure
For Teachers: Gamification isn't about trivializing learning—it's about creating engagement patterns that sustain daily practice. The streak counter and XP system provide measurable motivation that maintains student participation across weeks and months.
9 / 12

Security & Access Architecture

Enterprise Compliance
🔒
JWT Token Authentication
Stateless session management without server-side session storage. Supports horizontal scaling for thousands of concurrent users. Tokens include user claims and role information for access control.
🛡️
Multi-Tenant Isolation
School-level data segregation enforced at database query level via `schoolId` filtering. School A's data never appears in School B's dashboard. Supports the multi-tenant school deployment model.

Access Control Matrix

✓ Student Data Isolation
GDPR/FERPA compliant data segregation
✓ Teacher Dashboard Access
Aggregate class analytics without individual exposure
✓ Content Protection
Presigned URLs with configurable expiry for temporary access
✓ Audit Logging
Every action tracked for institutional compliance reporting
For Schools: We use the same security standards as major fintech applications. Schools can rest assured that student data is protected with enterprise-grade encryption and access controls.
10 / 12

Responsive Design & Cross-Device

Mobile-First Architecture
📱
Device Optimization

Desktop (≥1024px): Full feature set with keyboard/mouse optimization. Tablet (768-1023px): Touch-optimized layouts with 44px minimum targets. Mobile (<768px): Streamlined single-column with bottom-sheet patterns.

CSS Module Architecture

:global(.mobile) {
  font-size: 14px;
  padding: 0 16px;
}

:global(.tablet) {
  grid-template-columns: 1fr 1fr;
  gap: 16px;
}
🔄
State Synchronization
All learning progress synchronizes via Redux persisted to cloud. When switching devices mid-practice, students continue exactly where they left off. Conflict resolution: timestamp-based last-write-wins.
📴
Offline Capability
Critical content (vocabulary lists, recent sessions) caches locally. New words download for offline review; pronunciation audio caches on first play. Activity syncs automatically when connectivity returns.
Hong Kong Context: Students frequently access during MTR commutes on limited data plans. Mobile optimizations ensure seamless learning on any device.
11 / 12

The Complete Learning Ecosystem

Summary & Next Steps
👨‍🏫
For Teachers
  • ▸ Real-time engagement monitoring
  • ▸ Reduced preparation time
  • ▸ Multilingual classroom support
👩‍🎓
For Students
  • ▸ Personalized learning paths
  • ▸ Phoneme-level feedback precision
  • ▸ AI tutor available 24/7
🏫
For Schools
  • ▸ Enterprise-grade security (JWT, AES)
  • ▸ Thinkific LMS integration
  • ▸ Scalable cloud infrastructure

"Every word, analyzed sound by sound—turning every class into a reusable learning asset."

Teacher Account: test_teacher@seechange-edu.com | Password: Aa123456

12 / 12