PanelMerge is a secure, enterprise-grade web application for researchers and clinicians to easily combine, filter, and download gene lists from multiple sources, including Genomics England PanelApp, PanelApp Australia, and user-uploaded custom gene panels. Features comprehensive panel library management with version control, multi-format export capabilities, and advanced security features.

• KnowHow Bookmarks: Save/unsave articles to a personal reading list; bookmarks page at /knowhow/bookmarks; toggle button on the article view
• "Helpful" Reactions: Per-user thumbs-up reactions on articles; count shown on article cards and the article view; new "Most helpful" category sort option
• Article Tags: Free-text comma-separated tags; sky-blue # label pill badges on cards and article view; filtered view per tag at /knowhow/tags/<label>
• Related Articles: Up to 5 same-category articles listed at the bottom of each article view
• Category Descriptions: Category description text displayed on category detail pages
• Print / PDF Export: "Print" button on article view; @media print CSS gives a clean printout (hides nav, buttons, and UI chrome)
• "New since last visit" badge: White/red count badge on index category headers counts items added since the user's last visit to that category

• KnowHow Full-text Search: Search box on the KnowHow index; GET /knowhow/search?q= queries article titles, content, and link descriptions/URLs via ILIKE; highlighted snippets; up to 50 results per type; audit-logged
• KnowHow Category Detail Pages: Each category header on the index links to /knowhow/category/<slug> showing all articles and links for that category (no truncation)
• KnowHow Index Truncation: Index shows at most 3 most-recent articles per bucket; a "+ N more — see all" link appears when there are more
• KnowHow Category Sort: Sort selector (5 options: position, A→Z, Z→A, most content, recently updated) with cookie persistence
• KnowHow Article Summary Field: Optional 512-char plain-text teaser shown beneath article titles in index and category views; editor textarea added; summary DB column (nullable VARCHAR 512)

• GDPR Retention Controls: Admin-triggered purge routes for visit logs (90-day), suspicious activity records (90-day), and panel download logs (12-month); new deletion modals in the Admin panel
• Stored-XSS Protection (DPIA R12): KnowHow articles sanitized server-side with nh3 before storage — prevents malicious HTML/JS from executing in other users' browsers
• NCBI Transfer Disclosure (DPIA R7): Privacy Policy updated with an explicit Art. 49(1)(b) disclosure for PubMed queries routed to NCBI (USA); amber notice shown on the LitReview search page before submission
• LitReview Retention & Self-Service Deletion (DPIA R8): 365-day automated purge CLI (flask litreview cleanup); per-search delete and Clear All on the Search History page
• PanelGene Annotations Privacy Notices: Amber warning below the Gene List input and visibility hint on the panel modal reminding users not to include patient-identifiable data in gene notes
• KnowHow Content Warning (DPIA R11): Red banner added to the article editor warning against patient-identifiable content
• Privacy Policy v1.2: New sections 3.5 (Saved Panels), 3.6 (Security Infrastructure / geolocation), 3.7 (Exports & Download Logging); updated legal basis table and retention schedule
• DPIA updated to v1.4: All GDPR action items for v1.6 gaps resolved and ticked

• Dynamic KnowHow Categories: Admin-managed categories replace hardcoded sections — add/edit/remove categories with custom colours, descriptions, and ordering
• KnowHow Subcategories (Folders): Optional folder nesting within categories; articles and links assignable to subcategories
• KnowHow Admin UI: New /knowhow/admin page with hex colour picker and full CRUD for categories and subcategories
• Logout Fix: Session cookie now correctly cleared on logout (fixed ordering of destroy_session() / logout_user())
• Link Delete Button: Hover-reveal × button on all KnowHow links (owner or admin only)

• Saved Panel Library System:

  • Personal panel storage with modifications for future use
  • Git-like version control with configurable retention (default: 10 versions)
  • Tag system for important versions (e.g., "v1.0-production")
  • Branch/merge capabilities for panel evolution tracking
  • Visual version timeline with branch visualization
  • Google Cloud Storage integration with multi-backend support

• My Panels Profile Tab:

  • Comprehensive panel management interface with sortable grid
  • Advanced filtering by name, date, source, gene count, and sharing status
  • Quick actions for edit, export, share, and delete operations
  • Inline editing of panel metadata and gene lists
  • Real-time validation and error highlighting

• Multi-Format Export System:

  • Export panels in Excel (.xlsx), CSV, TSV, and JSON formats
  • Excel exports include multiple sheets (genes, metadata, version history)
  • Batch export functionality for multiple panels
  • Export Wizard with custom filenames and column selection
  • Export template creation for recurring export needs
  • Template management in user profile

• Enhanced Security Features:

  • Password history tracking to prevent password reuse
  • Account lockout protection after multiple failed attempts
  • Single-use password reset tokens with expiration
  • Admin password override with secure temporary passwords
  • Email change verification system
  • Suspicious activity detection with geographic anomaly analysis

• Advanced Filtering:

  • Multi-criteria panel filtering by status, version, date, and gene count
  • Save and reuse filter configurations
  • Filter presets for common search patterns

• Database Testing Suite:

  • Comprehensive 50+ test suite for schema validation
  • Data integrity and security testing
  • Migration testing for schema evolution
  • Multi-environment support (SQLite/PostgreSQL)

• LitReview Module (Preview):

  • New Literature Review blueprint for future development
  • Placeholder for PubMed integration and literature analysis
  • Accessible via Tools menu in navigation

• Comprehensive Security Audit Logging:

  • 33 audit action types including security violations, access denied events, and compliance logging
  • Real-time threat detection with automated response capabilities
  • Risk assessment scoring (0-100) for security events

• Enterprise-Grade Security Monitoring:

  • Automated detection of SQL injection, path traversal, and brute force attacks
  • Suspicious user agent detection and IP blocking
  • File upload security validation with malicious content detection
  • Rate limiting and behavioral anomaly detection

• Advanced Session Management:

  • Enhanced session security with individual session revocation
  • Redis-based session storage with secure token rotation
  • Session hijacking protection and privilege escalation monitoring

• Data Encryption & Compliance:

  • Complete data encryption at rest and in transit
  • GDPR compliance logging and regulatory event tracking
  • Comprehensive audit trail for forensic analysis

• PanelApp Integration:

  • Search and select gene panels from Genomics England PanelApp (UK) and PanelApp Australia.
  • Filter genes by rating (e.g., Green, Amber, Red) and disease group.
  • Search by panel name, description, disease group, or gene name (e.g., "BRCA1").
  • View panel details and gene counts before combining.

• Enhanced Search Capabilities:

  • Text-based search across panel names, descriptions, and disease groups.
  • Gene-based search to find panels containing specific genes.
  • Combined search results with duplicate removal.
  • Real-time filtering with debounced input.

• User Panel Upload:

  • Upload your own gene panels in Excel (.xls, .xlsx), CSV, or TSV format.
  • Flexible column naming: accepts "gene", "genes", "entity_name", or "genesymbol" (case-insensitive).
  • Drag-and-drop or click-to-select multiple files.
  • Prevents duplicate uploads and allows removal of files before and after upload.
  • Uploaded panels are stored per session and can be combined with PanelApp panels.

• Gene List Generation:

  • Combine selected PanelApp panels and user-uploaded panels into a single Excel file.
  • Each user-uploaded panel appears as a separate sheet in the Excel output.
  • The "Combined list" sheet includes all unique genes, with a column indicating the source panel(s), including user panel file names.

• Modern, User-Friendly UI:

  • Tabbed interface for UK, Australia, and Upload Panel workflows.
  • Real-time feedback on upload status, file list, and errors.
  • Responsive design using Tailwind CSS and Bootstrap (for admin pages).
  • Header navigation with version history tracking.

• Saved Panel Library:

  • Personal panel storage with complete version control
  • Share panels with other users and manage permissions
  • Comprehensive panel metadata tracking
  • Google Cloud Storage backend with local file system backup
  • Automatic versioning with optional commit messages

• My Panels Management:

  • Dedicated profile tab for managing saved panels
  • Visual version timeline with branch visualization
  • Advanced search and filtering capabilities
  • Inline editing with real-time validation
  • Quick actions for common operations

• Multi-Format Export:

  • Export panels in Excel, CSV, TSV, and JSON formats
  • Customizable export templates for recurring needs
  • Batch export for multiple panels simultaneously
  • Include metadata and version history in exports

• Admin Dashboard:

  • Login-protected admin area for managing users and viewing download logs
  • Site Messages System: Create and manage announcements displayed on the main page
    • Support for Info, Success, Warning, and Error message types with color coding
    • Optional expiration dates for automatic message removal
    • Live preview when creating messages
    • Toggle active/inactive status for immediate control
    • Full audit logging for all administrative actions
  • Account Management: Unlock locked accounts and manage security settings
  • Enhanced Audit Log Viewer: Advanced filtering and search capabilities

• Flexible Database Support:

  • Can run with or without database (set WITHOUT_DB=True in .env)
  • SQLite (local development) or Cloud SQL (production) supported
  • Support for multiple storage backends (GCS, local file system)

1. Search for Panels:

   • Use the search field to find panels by name, disease group, or gene name.
   • Examples: "BRCA1" (gene), "cardiac" (panel name), "heart disease" (description).

2. Select and Configure:

   • Choose panels from UK or Australian PanelApp using the tabbed interface.
   • Select gene confidence levels (Green, Amber, Red) for each panel.
   • Optionally upload your own gene panel files via the Upload Panel tab.

3. Generate Combined List:

   • Click "Generate Gene List" to download a combined Excel file.
   • Each source appears as a separate sheet with a combined summary sheet.

• Supported formats: .csv, .tsv, .xls, .xlsx.
• Required column: One of gene, genes, entity_name, or genesymbol (case-insensitive).
• Session-based: Uploaded files are stored per session and not shared between users.
• Multiple files: Upload multiple panels at once with duplicate prevention.

• Backend: Python, Flask, SQLAlchemy, Pandas, openpyxl, Redis
• Frontend: JavaScript, Tailwind CSS, Bootstrap (admin UI)
• Security: Enterprise encryption service, comprehensive audit logging, threat detection, account lockout
• Storage: Google Cloud Storage (primary), Local file system (backup), Multi-backend architecture
• APIs: Genomics England PanelApp, PanelApp Australia, Saved Panel Management API
• Database: PostgreSQL (production), SQLite (local/testing), Redis (caching/sessions)
• Build Tools: npm, Tailwind CSS compiler
• Testing: pytest, unittest, comprehensive database and API testing
• Deployment: Google Cloud Platform with Cloud SQL and Cloud Storage

• /api/panels?source={uk|aus} - Get all panels from specified source
• /api/genes/{entity_name}?source={uk|aus} - Find panels containing specific gene

• /upload_user_panel - Upload custom gene panels
• /uploaded_user_panels - List uploaded panels in session
• /remove_user_panel - Remove uploaded panel from session

• /api/user/panels - List user's saved panels
• /api/user/panels (POST) - Save new panel
• /api/user/panels/{id} - Get specific panel
• /api/user/panels/{id} (PUT) - Update panel
• /api/user/panels/{id} (DELETE) - Delete panel
• /api/user/panels/{id}/versions - List panel versions
• /api/user/panels/{id}/versions/{version} - Get specific version
• /api/user/panels/{id}/versions/{version}/restore - Restore version
• /api/user/panels/{id}/diff/{v1}/{v2} - Compare versions
• /api/user/panels/{id}/merge - Merge updates
• /api/user/panels/{id}/share - Share panel
• /api/user/panels/{id}/duplicate - Duplicate panel
• /api/user/panels/{id}/export/{format} - Export panel
• /api/user/panels/import - Import panel
• /api/shared/panels - List shared panels

• /api/version - Application version information

• /admin/messages - Admin message management
• /admin/messages/create - Create new site messages
• /admin/unlock-account - Unlock locked user accounts

# Install dependencies
npm install
pip install -r requirements.txt

# Build CSS
npm run build:css

# Run development server
python run.py

• Configure environment variables in .env
• Set WITHOUT_DB=True for database-free operation
• Use SQLITE_DB_PATH for local SQLite database
• Deploy to cloud with Google Cloud SQL for production

• Google Cloud PostgreSQL: See docs/GOOGLE_CLOUD_POSTGRESQL_SETUP.md for complete setup instructions
• Google Cloud Storage: See docs/GOOGLE_CLOUD_STORAGE_SETUP.md for storage backend configuration
• Quick Reference: See docs/POSTGRESQL_QUICK_REFERENCE.md for daily operations
• Storage Reference: See docs/STORAGE_QUICK_REFERENCE.md for storage operations
• Testing: Comprehensive database testing framework with 50+ test cases

• CHANGELOG.md - Complete version history and changes
• docs/FutureImprovements.txt - Feature roadmap and implementation status
• docs/UPDATE_CHECKLIST.md - Version update checklist and procedures

• docs/LITREVIEW_FEATURES.md - Literature Review feature specification
• docs/PROFILE_TEMPLATES_IMPLEMENTATION.md - Export template system
• docs/MY_PANELS_PROFILE_TAB.md - Panel library management
• docs/PANEL_EXPORT_SYSTEM.md - Multi-format export system
• docs/EXPORT_WIZARD.md - Export wizard documentation

• docs/SECURITY_GUIDE.md - Security implementation guide
• docs/PASSWORD_HISTORY_IMPLEMENTATION.md - Password security features
• docs/ACCOUNT_LOCKOUT_SYSTEM.md - Account lockout protection
• docs/PASSWORD_RESET_SYSTEM.md - Password reset security
• docs/EMAIL_CHANGE_VERIFICATION_IMPLEMENTATION.md - Email verification

• docs/GOOGLE_CLOUD_POSTGRESQL_SETUP.md - PostgreSQL database setup guide
• docs/GOOGLE_CLOUD_STORAGE_SETUP.md - Cloud Storage setup guide
• docs/POSTGRESQL_QUICK_REFERENCE.md - Database quick reference
• docs/STORAGE_QUICK_REFERENCE.md - Storage quick reference

• docs/TESTING_FRAMEWORK.md - Testing framework documentation
• Database testing suite with 50+ comprehensive tests
• API testing with authentication and authorization tests

MIT License