Reference Implementation: eda-front-end (Data Mining Frontend)¶
Overview¶
The EDA front-end is the web application for Nextpoint Data Mining. It provides a single-page application for managing document processing projects: importing batches, running searches, configuring search groups/slices, generating reports, exporting data, assigning custodians, and managing ML operations.
It also deploys its own ~50 Lambda functions that serve as the API middleware layer between the UI and DynamoDB/backend.
This is NOT an NGE service module. It is part of the separate Data Mining product.
Architecture¶
eda-front-end/
├── src/
│ ├── modules/ # Core SPA modules
│ │ ├── config.ts # Global config (injected at build time)
│ │ ├── api.ts # API client (API Gateway + Cognito auth)
│ │ ├── auth.ts # Cognito authentication
│ │ ├── router.ts # Hash-based SPA routing
│ │ └── s3.ts # Direct S3 access (Cognito Identity Pool)
│ ├── components/ # ~60+ Web Components
│ │ ├── *.html.hbs # Handlebars templates (define <template> elements)
│ │ └── *.ts # TypeScript classes (register custom elements)
│ ├── lambdas/ # ~50 Lambda functions (API middleware)
│ │ ├── batches/ # Batch CRUD
│ │ ├── custodians/ # Custodian management
│ │ ├── locations/ # Data source locations
│ │ ├── searches/ # Search management
│ │ ├── search-groups/ # Search group CRUD
│ │ ├── slices/ # Slice management
│ │ ├── users/ # User management
│ │ ├── projects/ # Project CRUD
│ │ ├── reports/ # Report management
│ │ ├── exports/ # Export management
│ │ ├── ml/ # ML operations
│ │ └── sfn-*/ # Step Functions handlers
│ └── pages/ # SPA pages
│ ├── home/ # Dashboard
│ ├── login/ # Cognito login
│ ├── project/ # Project management
│ ├── import/ # Batch import
│ ├── search/ # Search configuration
│ ├── slice/ # Slice management
│ ├── export/ # Export configuration
│ ├── settings/ # Settings
│ └── debug/ # Debug tools
├── scripts/deploy/ # Custom Node.js deployment system
│ ├── cognito.js # Cognito User Pool, Client, Identity Pool
│ ├── api-gateway.js # API Gateway creation
│ ├── lambda.js # Lambda function deployment
│ ├── dynamodb.js # DynamoDB table creation
│ ├── iam.js # IAM role management
│ ├── s3.js # S3 bucket setup
│ ├── cloudfront.js # CloudFront distribution
│ ├── route53.js # DNS management
│ ├── kms.js # KMS key management
│ ├── amplify.js # AWS Amplify hosting
│ └── step-functions.js # Step Functions state machines
├── amplify.yml # AWS Amplify build configuration
└── package.json # Dependencies and scripts
Stack¶
- Frontend: TypeScript, Web Components (custom elements), Handlebars templates, Sass
- Build: esbuild (bundler), Sass
- Hosting: AWS Amplify
- Auth: Amazon Cognito (User Pools + Identity Pools)
- Data: DynamoDB (single-table design: pk/sk/gsipk/gsisk)
- API: API Gateway (REST) with Cognito authorizer → ~50 Lambda functions (Node.js 20.x)
- S3: Direct browser access via Cognito Identity Pool credentials
- UI Libraries: AG Grid (enterprise), Chart.js, Flatpickr, SweetAlert2, jsPDF, PapaParse
- Deploy: Custom Node.js scripts (NOT CDK)
Key Design Decisions¶
Web Components (No Framework)¶
Uses native Web Components (custom elements) instead of React/Vue/Angular:
- Each component = .html.hbs template + .ts class extending HTMLElement
- Templates use Handlebars for basic templating
- No virtual DOM, no framework overhead
- Direct DOM manipulation via this.shadowRoot
DynamoDB Single-Table Design¶
One project table with composite keys:
- pk / sk — primary partition/sort key
- gsipk / gsisk — GSI for alternate access patterns
- DynamoDB Streams trigger Lambda functions for cascading updates
Own API Middleware Layer¶
The front-end repo deploys ~50 Lambda functions that serve as the API between the UI and both DynamoDB (project metadata) and the EDA backend (processing). This is unusual — most SPAs call the backend API directly.
Direct S3 Browser Access¶
Uses Cognito Identity Pool to get temporary AWS credentials in the browser, allowing direct S3 access for browsing document locations without proxying through an API.
Integration Points¶
- EDA backend: Connected via
DM_BACKEND_ENV_NAMEconfig. Front-end's API Gateway calls backend API Gateway or invokes backend Lambda functions. - Cognito: Own User Pool, Client, and Identity Pool for auth
- S3: Shared data buckets with EDA backend
- CORS: Front-end deploy sets
CORS_ALLOWED_ORIGINon backend Lambda functions
Key Differences from NGE Architecture¶
| Aspect | EDA Front-End | NGE / Rails |
|---|---|---|
| UI framework | Web Components + Handlebars | Rails ERB + React |
| Auth | Cognito (own User Pool) | Rails sessions + Cognito (shared) |
| Data store | DynamoDB (single-table) | MySQL (per-case) |
| API | Own Lambda middleware layer | Rails controllers |
| Hosting | AWS Amplify | EC2 / ECS |
| Deploy | Custom Node.js scripts | CDK / Capistrano |
Key File Locations¶
| File | Purpose |
|---|---|
src/modules/config.ts |
Global configuration constants |
src/modules/api.ts |
API client with Cognito auth |
src/modules/auth.ts |
Cognito authentication |
src/modules/router.ts |
Hash-based SPA routing |
src/modules/s3.ts |
Direct S3 browser access |
src/lambdas/ |
~50 Lambda API functions |
src/components/ |
~60+ Web Components |
scripts/deploy/ |
Custom deployment system |
amplify.yml |
Amplify build config |
Ask questions about Nextpoint architecture, patterns, rules, or any module. Powered by Claude Opus 4.6.