📘 Free CERTIFIED-GENERATIVE-AI-ENGINEER-ASSOCIATE Sample Questions
A Generative Al Engineer has created a RAG application to look up answers to questions about a series of fantasy
novels that are being asked on the author’s web forum. The fantasy novel texts are chunked and embedded into a
vector store with metadata (page number, chapter number, book title), retrieved with the user’s query, and provided
to an LLM for response generation. The Generative AI Engineer used their intuition to pick the chunking strategy and
associated configurations but now wants to more methodically choose the best values.
Which TWO strategies should the Generative AI Engineer take to optimize their chunking strategy and parameters?
(Choose two.)
A
Change embedding models and compare performance
B
Add a classifier for user queries that predicts which book will best contain the answer. Use this to filter
retrieval.
C
Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the
chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the best
performance metric.
D
Pass known questions and best answers to an LLM and instruct the LLM to provide the best token count.
Use a summary statistic (mean, median, etc.) of the best token counts to choose chunk size.
E
Create an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most
appropriate chunk. Optimize the chunking parameters based upon the values of the metric.
Correct Answer:
C. Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the
chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the best
performance metric.
A Generative AI Engineer is designing a RAG application for answering user questions on technical regulations as they
learn a new sport.
What are the steps needed to build this RAG application and deploy it?
A
Ingest documents from a source –> Index the documents and saves to Vector Search –> User submits
queries against an LLM –> LLM retrieves relevant documents –> Evaluate model –> LLM generates a response
–> Deploy it using Model Serving
B
Ingest documents from a source –> Index the documents and save to Vector Search –> User submits
queries against an LLM –> LLM retrieves relevant documents –> LLM generates a response -> Evaluate model
–> Deploy it using Model Serving
C
Ingest documents from a source –> Index the documents and save to Vector Search –> Evaluate model –>
Deploy it using Model Serving
D
User submits queries against an LLM –> Ingest documents from a source –> Index the documents and save
to Vector Search –> LLM retrieves relevant documents –> LLM generates a response –> Evaluate model –>
Deploy it using Model Serving
Correct Answer:
B. Ingest documents from a source –> Index the documents and save to Vector Search –> User submits
queries against an LLM –> LLM retrieves relevant documents –> LLM generates a response -> Evaluate model
–> Deploy it using Model Serving
A Generative AI Engineer just deployed an LLM application at a digital marketing company that assists with answering
customer service inquiries.
Which metric should they monitor for their customer service LLM application in production?
A
Number of customer inquiries processed per unit of time
B
Energy usage per query
C
Final perplexity scores for the training of the model
Correct Answer:
A. Number of customer inquiries processed per unit of time
A Generative AI Engineer is building a Generative AI system that suggests the best matched employee team member
to newly scoped projects. The team member is selected from a very large team. The match should be based upon
project date availability and how well their employee profile matches the project scope. Both the employee profile
and project scope are unstructured text.
How should the Generative Al Engineer architect their system?
A
Create a tool for finding available team members given project dates. Embed all project scopes into a
vector store, perform a retrieval using team member profiles to find the best team member.
B
Create a tool for finding team member availability given project dates, and another tool that uses an LLM
to extract keywords from project scopes. Iterate through available team members’ profiles and perform
keyword matching to find the best available team member.
C
Create a tool to find available team members given project dates. Create a second tool that can calculate a
similarity score for a combination of team member profile and the project scope. Iterate through the team
members and rank by best score to select a team member.
D
Create a tool for finding available team members given project dates. Embed team profiles into a vector
store and use the project scope and filtering to perform retrieval to find the available best matched team
members.
Correct Answer:
D. Create a tool for finding available team members given project dates. Embed team profiles into a vector
store and use the project scope and filtering to perform retrieval to find the available best matched team
members.
A Generative AI Engineer is designing an LLM-powered live sports commentary platform. The platform provides realtime updates and LLM-generated analyses for any users who would like to have live summaries, rather than reading a
series of potentially outdated news articles.
Which tool below will give the platform access to real-time data for generating game analyses based on the latest
game scores?
A
DatabricksIQ
B
Foundation Model APIs
C
Feature Serving
D
AutoML
Correct Answer:
C. Feature Serving
A Generative AI Engineer has a provisioned throughput model serving endpoint as part of a RAG application and
would like to monitor the serving endpoint’s incoming requests and outgoing responses. The current approach is to
include a micro-service in between the endpoint and the user interface to write logs to a remote server.
Which Databricks feature should they use instead which will perform the same task?
A
Vector Search
B
Lakeview
C
DBSQL
D
Inference Tables
Correct Answer:
D. Inference Tables
A Generative Al Engineer is tasked with improving the RAG quality by addressing its inflammatory outputs.
Which action would be most effective in mitigating the problem of offensive text outputs?
A
Increase the frequency of upstream data updates
B
. Inform the user of the expected RAG behavior
C
Restrict access to the data sources to a limited number of users
D
Curate upstream data properly that includes manual review before it is fed into the RAG system
Correct Answer:
D. Curate upstream data properly that includes manual review before it is fed into the RAG system
A Generative Al Engineer is creating an LLM-based application. The documents for its retriever have been chunked to
a maximum of 512 tokens each. The Generative Al Engineer knows that cost and latency are more important than
quality for this application. They have several context length levels to choose from.
Which will fulfill their need?
A
context length 514; smallest model is 0.44GB and embedding dimension 768
B
context length 2048: smallest model is 11GB and embedding dimension 2560
C
context length 32768: smallest model is 14GB and embedding dimension 4096
D
context length 512: smallest model is 0.13GB and embedding dimension 384
Correct Answer:
D. context length 512: smallest model is 0.13GB and embedding dimension 384
Which of the following considerations is most important when creating and querying a Vector Search index for use in
a Generative AI application in Databricks?
A
Choose a vector indexing method optimized for high-dimensional data and ensure it supports efficient
similarity search operations
B
Use a SQL-based search engine to ensure the embeddings can be queried using standard SQL queries.
C
Store the embeddings in a CSV format for easier querying and storage within Databricks.
D
Ensure the document corpus is indexed in a relational database before creating vector embeddings.
Correct Answer:
A. Choose a vector indexing method optimized for high-dimensional data and ensure it supports efficient
similarity search operations
Explanation:
When working with Generative AI applications in Databricks that require vector search, it is crucial to use an
indexing method that is optimized for high-dimensional data. Embeddings used in such models are typically highdimensional vectors, and the search needs to be efficient in terms of both speed and accuracy. Using a vector
indexing method such as FAISS or Annoy, which are specifically designed for similarity search in high-dimensional
spaces, ensures that the application can perform efficiently. Other methods like relational databases or CSV
formats would not be optimized for this purpose and would result in slower and less efficient querying.
You have successfully trained a machine learning model in Databricks using MLflow. Your next task is to register
the model to Unity Catalog for easy discovery and management. What are the correct steps you should take to
ensure the model is properly registered? (Select two)
A
Tag the model with a Unity Catalog-specific tag using mlflow.set_tag() before registering it
B
Use the Databricks Model Registry to register the model and select "Unity Catalog" as the destination.
C
Register the model manually by navigating to the Unity Catalog tab in the Databricks workspace.
D
Set the environment variable MLFLOW_MODEL_REGISTRY_URI to the Unity Catalog URI before running your
MLflow script.
E
Use the MLflow mlflow.register_model() function with the Unity Catalog URI.
Correct Answer:
B. Use the Databricks Model Registry to register the model and select "Unity Catalog" as the destination.
Questions: 1-10 out of 127
Continue Full Practice..
GET ALL 127 QUESTIONS