Enhancing Legal Consultation Services with GenAI and LLMs
8/27/2024
Background
The legal industry has seen a significant shift towards online platforms, offering accessible and reliable legal advice to clients. An online lawyer consultation platform sought to leverage this trend to expand its client base and improve service delivery. The platform not only provided legal advice but also offered a lawyer matching service to connect clients with suitable legal professionals. However, maintaining a high level of service quality posed challenges, particularly due to the labor-intensive nature of providing legal consultations and matching clients with the right lawyers. To address these challenges, the platform aimed to implement AI to deliver faster and more accurate responses to common legal inquiries, optimize the lawyer matching process, and reduce operational costs.
Challenges
The platform faced several obstacles in adopting AI technology:
Budget Constraints: Developing an edge Large Language Model (LLM) training and fine-tuning server required a substantial investment, which was a significant barrier for the platform.
Security Concerns: Training and fine-tuning LLMs using cloud services raised security concerns, particularly because the platform handled sensitive and confidential legal data.
Infrastructure Limitations: Law firms typically lack the necessary server racks, and the implementation of an AI solution required resolving power source and noise issues associated with high-performance computing equipment.
Advantech Solutions
To overcome these challenges, the customer chose the AIR-520 Llama-3 70B LLM fine-tuning solution, which includes the following components:
AIR-520 Edge AI Server System with NVIDIA RTX 4000 Ada GPU Cards: This server-grade system, equipped with four NVIDIA RTX 4000 Ada GPU cards, provided the computational power needed for AI processing, allowing the platform to train and fine-tune models in-house.
aiDAPTIV+ ai100 AI SSD: The AIR-520 is equipped with the ai100 SSD, featuring 4TB of storage and patented middleware, facilitating efficient data management and model training, ensuring that the platform can handle large datasets securely and effectively.
No-Code GUI for Model Fine-Tuning: A user-friendly, no-code graphical user interface (GUI) enabled the customer to fine-tune AI models with local language datasets, enhancing the model's relevance and accuracy for different regions.
Benefits
- Localized Knowledge Base: Different countries have different laws and languages. The more the model learns about localized knowledge, the higher the quality of the generated content users will receive.
- Ease of Deployment: The compact tower system is suitable for firm offices. Users do not need to learn Linux command lines or parameters, thanks to the system's built-in no-code GUI for model fine-tuning and validation.
- Expanded Business Opportunities: This industrial solution ensures 24/7 online chatbot service and prompt assistance, attracting more potential clients and lawyers to the platform and enhancing loyalty.
-25/EPC-B5000_Front-left--_Square20231115180815.jpg)
-25/AIR-520_04--20240503141558.jpg)
-25/PR-AIR-520_w820h46020240830105212.jpg)
-25/AIR-510_04--20240806132906.jpg)
