University of British Columbia Cloud Innovation Centre: Prototyping generative AI solutions using AWS

The University of British Columbia (UBC) Cloud Innovation Centre (CIC) has become a hub for innovation by prototyping generative AI applications in collaboration with public sector sponsors. Using Amazon Bedrock, the UBC CIC addresses real-world challenges through rapid prototyping. This post highlights how the UBC CIC uses Amazon Web Services (AWS) to accelerate generative AI development, sharing lessons learned, tools used, and actionable insights you can apply to your projects.

Importance of rapid prototyping in AI development

Rapid prototyping is critical in AI development. AI applications—particularly those built on large language models (LLMs)—require frequent iterations to refine prompts, adjust data pipelines, and optimize outputs. With more than 55 solutions produced, the UBC CIC understands that the faster teams can develop prototypes, the sooner sponsors can see results and provide feedback. At the UBC CIC, students have used Amazon Bedrock to build more than seven generative AI solutions, saving dozens of hours in setting up and maintaining infrastructure.

Amazon Bedrock is a managed service that offers a choice of models, both open source and from leading AI companies, through a single API. Amazon Bedrock offers a broad set of capabilities needed to build generative AI applications with security, privacy, and responsible AI.

Security in generative AI prototyping

UBC CIC observes the shared responsibility model through the Amazon Bedrock Data protection features. Amazon Bedrock doesn’t store or share customer data or use it to train AWS models. This provides privacy, aligning with UBC CIC’s commitment to data security for their projects.

UBC CIC uses AWS Identity and Access Management (IAM) for access control, limiting developers and users to specific models and resources. For instance, UBC CIC can limit access to prevent unauthorized Provisioned Throughput during development or to restrict the use of models that don’t align with project requirements. Regular AWS security updates help build secure generative AI applications while protecting user data.

The UBC CIC uses Amazon Bedrock Guardrails for content safety. Amazon Bedrock filters specific topics, blocks harmful content, and redacts personally identifiable information (PII) in inputs and outputs. These features are essential for building secure generative AI applications.

The following diagram illustrates the Amazon Bedrock Guardrails workflow that UBC CIC uses for content safety.

Figure 1. Amazon Bedrock Guardrails workflow used by the UBC CIC for content safety

Selecting the right AWS services for generative AI prototyping

UBC CIC’s ability to develop high-quality innovations stems from its use of AWS services, which means that students can focus on development, confident that they can implement appropriate privacy and security measures, without having to spend time on infrastructure. Key services include:

• Amazon Bedrock – Offers foundation models from Anthropic, Meta, AI21 Labs, Mistral and Stability AI
• Amazon Relational Database Service (Amazon RDS): Stores embeddings for Retrieval Augmented Generation (RAG) using pgvector extension
• Amazon DynamoDB: Manages chat histories
• AWS Lambda: Runs backend logic
• Amazon API Gateway: Exposes APIs for the generative AI applications
• AWS Amplify: Streamlines frontend deployment

The following diagram illustrates the UBC CIC reference architecture for generative AI solutions.

Figure 2. UBC CIC reference architecture diagram for generative AI solutions. The major components are AWS Amplify, Amazon API Gateway, AWS Lambda, Amazon Bedrock, and Amazon RDS

Amazon Bedrock

Amazon Bedrock empowers generative AI prototypes with straightforward testing and deployment. It provides access to a range of LLMs, including Meta’s Llama and Mistral AI. This helps UBC CIC students focus on solving sponsor problems instead of managing infrastructure.

LLMs have been evolving rapidly, and Amazon Bedrock has consistently enabled new versions with high frequency. This continuous advancement has empowered the UBC CIC to evolve its solutions, enabling them to remain at the forefront of AI-driven innovation.

Serverless architecture

By opting for a serverless architecture, the UBC CIC demonstrates:

• Cost efficiency: With UBC CIC prototypes sponsors only pay for used resources
• Scalability: Amazon Bedrock, Lambda, and DynamoDB adjust to usage demands
• Faster time to market: UBC CIC iterates in 2-week development cycles, showing rapid progress to sponsors

Using LangChain

UBC CIC implements LangChain for generative AI applications. LangChain’s abstraction layer unifies LLM interactions, speeding up development and testing and making updates easier as models evolve. The framework supports scalable, production-ready generative AI applications.

Evaluating and selecting LLMs

UBC CIC selects LLMs by assessing problem domains and finding the smallest effective models. They test pre-trained models in Amazon Bedrock for each use case. Clear project goals guide evaluations. A/B testing with predefined prompts compares LLM performance.

Students use Amazon SageMaker notebooks to refine prompts. They move validated prompts into the prototype using Lambda for processing and API Gateway for model interactions. SageMaker integration with the AWS console creates a streamlined pipeline from testing to deployment, providing flexibility and precision.

RAG and vector storage

UBC CIC uses pgvector with Amazon RDS for PostgreSQL to store and search vectors. This extension enables similarity searches across text embeddings. It offers cost-effective vector storage in SQL, combining fast searches with relational database features.

Setting up the environment

UBC CIC uses infrastructure as code (IaC) with AWS Cloud Development Kit (AWS CDK). Students code infrastructure in TypeScript, enabling fast creation and changes. This provides consistency, version control, and simple replication for generative AI prototypes. Students can deploy solutions in an average of 2 hours instead of days, reducing errors and letting them focus on development.

UBC CIC generative AI projects prototyped on AWS

The UBC Faculty of Science Advising Office collaborated with UBC CIC to develop a Student Advising Assistant, shown in the following screenshot. During phase one, prior to Amazon Bedrock release, they used an LLM hosted on Amazon Elastic Compute Cloud (Amazon EC2), running costs constantly. In the second phase, UBC CIC switched to Amazon Bedrock with Meta’s Llama 3.1 8B model, improving response quality while reducing costs by more than 50 percent.

Figure 3. The Student Advising Assistant prototype developed by the UBC CIC

UBC CIC and the UBC Centre for Teaching, Learning and Technology created AI Learning Assistant (AILA). This prototype encourages students to answer questions based on course materials, tracks comprehension, and identifies knowledge gaps. Administrators manage courses, assign instructors, and control access. The serverless app, shown in the following three screenshots, uses Amazon Bedrock with Meta’s Llama 3 70B to provide conversational guidance, supporting students’ academic progress.

Figure 4. The student view of the AILA prototype developed by the UBC CIC

Figure 5. Progress tracking in the AILA student view

Figure 6. The instructor view of the AILA prototype developed by the UBC CIC

Conclusion

Prototyping generative AI applications on AWS using Amazon Bedrock has empowered the UBC CIC to solve real-world challenges with speed and precision. Serverless architectures and tools like AWS CDK have helped teams to minimize infrastructure management, focusing instead on data preparation and prompt engineering.

Security remains a cornerstone of this workflow. With Amazon Bedrock, users retain full control over their data and knowledge bases, securely managing storage for embeddings in RAG setups and restricting model access through fine-grained IAM permissions, making it a reliable choice for organizations handling critical information.

To learn more, visit the UBC CIC portal or explore Amazon Bedrock for guidance on building secure and scalable generative AI applications.