Databricks Certified Professional Data Engineer Exam: Databricks-Certified-Professional-Data-Engineer Exam Premium File as PDF & Test Engine

Which REST API call can be used to review the notebooks configured to run as tasks in a multi-task job?

• A. /jobs/get
• B. /jobs/runs/get
• C. /jobs/list
• D. /jobs/runs/list
• E. /jobs/runs/get-output

Explanation: Only visible for Fast2test members. You can sign-up / login (it's free).

The data science team has created and logged a production using MLFlow. The model accepts a list of column names and returns a new column of type DOUBLE.
The following code correctly imports the production model, load the customer table containing the customer_id key column into a Dataframe, and defines the feature columns needed for the model.

Which code block will output DataFrame with the schema ' ' customer_id LONG, predictions DOUBLE ' ' ?

• A. Df, map (lambda k:midel (x [columns]) ,select ( ' ' customer_id predictions ' ' )
• B. Df. Select ( ' ' customer_id ' ' .
  Model ( ' ' columns) alias ( ' ' predictions ' ' )
• C. Df.apply(model, columns). Select ( ' ' customer_id, prediction ' '
• D. Model, predict (df, columns)

Explanation: Only visible for Fast2test members. You can sign-up / login (it's free).

A data engineer is designing a pipeline in Databricks that processes records from a Kafka stream where late- arriving data is common.
Which approach should the data engineer use?

• A. Implement a custom solution using Databricks Jobs to periodically reprocess all historical data.
• B. Use an Auto CDC pipeline with batch tables to simplify late data handling.
• C. Use batch processing and overwrite the entire output table each time to ensure late data is incorporated correctly.
• D. Use a watermark to specify the allowed lateness to accommodate records that arrive after their expected window, ensuring correct aggregation and state management.

Explanation: Only visible for Fast2test members. You can sign-up / login (it's free).

A task orchestrator has been configured to run two hourly tasks. First, an outside system writes Parquet data to a directory mounted at /mnt/raw_orders/. After this data is written, a Databricks job containing the following code is executed:
(spark.readStream
.format( " parquet " )
.load( " /mnt/raw_orders/ " )
.withWatermark( " time " , " 2 hours " )
.dropDuplicates([ " customer_id " , " order_id " ])
.writeStream
.trigger(once=True)
.table( " orders " )
)
Assume that the fields customer_id and order_id serve as a composite key to uniquely identify each order, and that the time field indicates when the record was queued in the source system. If the upstream system is known to occasionally enqueue duplicate entries for a single order hours apart, which statement is correct?

• A. The orders table will not contain duplicates, but records arriving more than 2 hours late will be ignored and missing from the table.
• B. Duplicate records enqueued more than 2 hours apart may be retained and the orders table may contain duplicate records with the same customer_id and order_id.
• C. All records will be held in the state store for 2 hours before being deduplicated and committed to the orders table.
• D. The orders table will contain only the most recent 2 hours of records and no duplicates will be present.

Explanation: Only visible for Fast2test members. You can sign-up / login (it's free).

A Delta Lake table with Change Data Feed (CDF) enabled in the Lakehouse named customer_churn_params is used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources. The churn prediction model used by the ML team is fairly stable in production. The team is only interested in making predictions on records that have changed in the past 24 hours. Which approach would simplify the identification of these changed records?

• A. Replace the current overwrite logic with a MERGE statement to modify only those records that have changed; write logic to make predictions on the changed records identified by the Change Data Feed.
• B. Convert the batch job to a Structured Streaming job using the complete output mode; configure a Structured Streaming job to read from the customer_churn_params table and incrementally predict against the churn model.
• C. Apply the churn model to all rows in the customer_churn_params table, but implement logic to perform an upsert into the predictions table that ignores rows where predictions have not changed.
• D. Modify the overwrite logic to include a field populated by calling current_timestamp() as data are being written; use this field to identify records written on a particular date.

Explanation: Only visible for Fast2test members. You can sign-up / login (it's free).