📘 Free PROFESSIONAL-DATA-ENGINEER Sample Questions
Your company built a TensorFlow neutral-network model with a large number of neurons and layers. The model fits well for the training data. However, when tested against new data, it performs poorly. What method can you employ to address this?
A
Threading
B
Serialization
C
Dropout Methods
D
Dimensionality Reduction
Correct Answer:
C. Dropout Methods
Explanation:
Bad performance of a model is either due to lack of relationship between dependent and independent variables used, or just overfit due to having used too many features and/or bad features.
A: Threading parallelisation can reduce training time, but if the selected featuers are the same then the resulting performance won't have changed
B: Serialization is only changing data into byte streams. This won't be useful.
C: This can show which features are bad. E.g. if it is one feature causing bad performance, then the dropout method will show it, so you can remove it from the model and retrain it.
D: This would become clear if the model did not fit the training data well. But the question says that the model fits the training data well, so D is not the answer.
Reference:
https://medium.com/mlreview/a-simple-deep-learning-model-for-stock-price-prediction-using-tensorflow- 30505541d877
You are building a model to make clothing recommendations. You know a user's fashion preference is likely to change over time, so you build a data pipeline to stream new data back to the model as it becomes available. How should you use this data to train the model?
A
Continuously retrain the model on just the new data.
B
Continuously retrain the model on a combination of existing data and the new data.
C
Train on the existing data while using the new data as your test set.
D
Train on the new data while using the existing data as your test set.
Correct Answer:
B. Continuously retrain the model on a combination of existing data and the new data.
Explanation:
As new data can be with new features. Hence the new data can be split to both training and test data to retrain as well as with existing data. because we have to use a combination of old and new test data as well as training data.
You designed a database for patient records as a pilot project to cover a few hundred patients in three clinics. Your design used a single database table to represent all patients and their visits, and you used self-joins to generate reports. The server resource utilization was at 50%. Since then, the scope of the project has expanded. The database must now store 100 times more patient records. You can no longer run the reports, because they either take too long or they encounter errors with insufficient compute resources. How should you adjust the database design?
A
Add capacity (memory and disk space) to the database server by the order of 200.
B
Shard the tables into smaller ones based on date ranges, and only generate reports with prespecified date ranges.
C
Normalize the master patient-record table into the patient table and the visits table, and create other necessary tables to avoid self-join.
D
Partition the table into smaller tables, with one for each clinic. Run queries against the smaller table pairs, and use unions for consolidated reports.
Correct Answer:
C. Normalize the master patient-record table into the patient table and the visits table, and create other necessary tables to avoid self-join.
Explanation:
Based on Google documentation, self-join is an anti-pattern because this option provides the least amount of inconvenience over using pre-specified date ranges or one table per clinic while also increasing performance due to avoiding self-joins.
Reference:
https://cloud.google.com/bigquery/docs/best-practices-performance-patterns
You create an important report for your large team in Google Data Studio 360. The report uses Google BigQuery as its data source. You notice that visualizations are not showing data that is less than 1 hour old. What should you do?
A
Disable caching by editing the report settings.
B
Disable caching in BigQuery by editing table details.
C
Refresh your browser tab showing the visualizations.
D
Clear your browser history for the past hour then reload the tab showing the virtualizations
Correct Answer:
A. Disable caching by editing the report settings.
Explanation:
Disable caching by editing the report settings.
A cache is a temporary data storage system. Fetching cached data can be much faster than fetching it directly from the underlying data set, and helps reduce the number of queries sent, minimizing costs for paid data access.
Reference:
https://support.google.com/datastudio/answer/7020039?hl=en#zippy=%2Cin-this-article https://support.google.com/datastudio/answer/7020039?hl=en
An external customer provides you with a daily dump of data from their database. The data flows into Google Cloud Storage GCS as comma-separated values
(CSV) files. You want to analyze this data in Google BigQuery, but the data could have rows that are formatted incorrectly or corrupted. How should you build this pipeline?
A
Use federated data sources, and check data in the SQL query.
B
Enable BigQuery monitoring in Google Stackdriver and create an alert.
C
Import the data into BigQuery using the gcloud CLI and set max_bad_records to 0.
D
Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.
Correct Answer:
D. Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.
Explanation:
Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.
By running a Cloud Dataflow pipeline to import the data, you can perform data validation, cleaning and transformation before it gets loaded into BigQuery. Dataflow allows you to handle corrupted or incorrectly formatted rows by pushing them to another dead-letter table for analysis. This way, you can ensure that only clean and correctly formatted data is loaded into BigQuery for analysis.
Your weather app queries a database every 15 minutes to get the current temperature. The frontend is powered by Google App Engine and server millions of users. How should you design the frontend to respond to a database failure?
A
Issue a command to restart the database servers.
B
Retry the query with exponential backoff, up to a cap of 15 minutes.
C
Retry the query every second until it comes back online to minimize staleness of data.
D
Reduce the query frequency to once every hour until the database comes back online.
Correct Answer:
B. Retry the query with exponential backoff, up to a cap of 15 minutes.
Explanation:
App engine create applications that use Cloud SQL database connections effectively. Below is what is written in google cloud documentation.
If your application attempts to connect to the database and does not succeed, the database could be temporarily unavailable. In this case, sending too many simultaneous connection requests might waste additional database resources and increase the time needed to recover. Using exponential backoff prevents your application from sending an unresponsive number of connection requests when it can't connect to the database.
This retry only makes sense when first connecting, or when first grabbing a connection from the pool. If errors happen in the middle of a transaction, the application must do the retrying, and it must retry from the beginning of a transaction. So even if your pool is configured properly, the application might still see errors if connections are lost.
Reference:
https://cloud.google.com/sql/docs/mysql/manage-connections
You are creating a model to predict housing prices. Due to budget constraints, you must run it on a single resource- constrained virtual machine. Which learning algorithm should you use?
A
Linear regression
B
Logistic classification
C
Recurrent neural network
D
Feedforward neural network
Correct Answer:
A. Linear regression
Explanation:
A tip here to decide when a liner regression should be used or logistics regression needs to be used. If you are forecasting that is the values in the column that you are predicting is numeric, it is always liner regression. If you are classifying, that is buy or no buy, yes or no, you will be using logistics regression.
You are building new real-time data warehouse for your company and will use Google BigQuery streaming inserts. There is no guarantee that data will only be sent in once but you do have a unique ID for each row of data and an event timestamp. You want to ensure that duplicates are not included while interactively querying data. Which query type should you use?
A
Include ORDER BY DESK on timestamp column and LIMIT to 1.
B
Use GROUP BY on the unique ID column and timestamp column and SUM on the values.
C
Use the LAG window function with PARTITION by unique ID along with WHERE LAG IS NOT NULL.
D
Use the ROW_NUMBER window function with PARTITION by unique ID along with WHERE row equals 1.
Correct Answer:
D. Use the ROW_NUMBER window function with PARTITION by unique ID along with WHERE row equals 1.
Explanation:
Description: Row Number equals 1 with partitioning will ensure only one record is fetched per partition
Your company is using WILDCARD tables to query data across multiple tables with similar names. The SQL statement is currently failing with the following error:
A
'bigquery-public-data.noaa_gsod.gsod'
B
bigquery-public-data.noaa_gsod.gsod*
C
'bigquery-public-data.noaa_gsod.gsod'*
D
'bigquery-public-data.noaa_gsod.gsod*`
Correct Answer:
D. 'bigquery-public-data.noaa_gsod.gsod*`
Explanation:
Reference:
https://cloud.google.com/bigquery/docs/wildcard-tables " target="_blank" style="word-break: break-all;">
Your company is in a highly regulated industry. One of your requirements is to ensure individual users have access only to the minimum amount of information required to do their jobs. You want to enforce this requirement with Google BigQuery. Which three approaches can you take? (Choose three.)
A
Disable writes to certain tables.
B
Restrict access to tables by role.
C
Ensure that the data is encrypted at all times.
D
Restrict BigQuery API access to approved users.
E
Segregate data across multiple tables or databases.
F
Use Google Stackdriver Audit Logging to determine policy violations.
Correct Answer:
B. Restrict access to tables by role.
Explanation:
bigquery.tables.create Create new tables. bigquery.tables.delete Delete tables. bigquery.tables.export Export table data out of BigQuery. bigquery.tables.get Get table metadata.
To get table data, you need bigquery.tables.getData.
bigquery.tables.getData Get table data. This permission is required for querying table data. To get table metadata, you need bigquery.tables.get.
bigquery.tables.list List tables and metadata on tables. bigquery.tables.setCategory Set policy tags in table schema. bigquery.tables.update
Update table metadata.
To update table data, you need bigquery.tables.updateData.
bigquery.tables.updateData Update table data.
To update table metadata, you need bigquery.tables.update.
Questions: 1-10 out of 389
Continue Full Practice..
GET ALL 389 QUESTIONS