Databricks Certified Professional Data Engineer - Databricks-Certified-Professional-Data-Engineer Exam Practice Test

A. No: the Delta Lake DELETE command only provides ACID guarantees when combined with the MERGE INTO command

B. No: the change data feed only tracks inserts and updates not deleted records.

C. Yes: Delta Lake ACID guarantees provide assurance that the DELETE command successed fully and permanently purged these records.

D. No: files containing deleted records may still be accessible with time travel until a BACUM command is used to remove invalidated data files.

A. resources:
jobs:
my-job:
name: data-pipeline
tasks: [...]
job_clusters: [...]
permissions:
- group_name: data-engineers
level: CAN_MANAGE
- group_name: auditors
level: CAN_VIEW

B. resources:
jobs:
my-job:
name: data-pipeline
tasks: [...]
job_clusters: [...]
permissions:
- group_name: data-engineers
level: CAN_MANAGE
- group_name: auditors
level: CAN_VIEW
- group_name: admin-team
level: IS_OWNER

C. resources:
jobs:
my-job:
name: data-pipeline
tasks: [...]
job: [...]
permissions:
- group_name: data-engineers
level: CAN_MANAGE
permissions:
- group_name: auditors
level: CAN_VIEW

D. permissions:
- group_name: data-engineers
level: CAN_MANAGE
- group_name: auditors
level: CAN_VIEW
resources:
jobs:
my-job:
name: data-pipeline
tasks: [...]
job_clusters: [...]

A. Skew caused by more data being assigned to a subset of spark-partitions.

B. Spill resulting from attached volume storage being too small.

C. Network latency due to some cluster nodes being in different regions from the source data

D. Task queueing resulting from improper thread pool assignment.

E. Credential validation errors while pulling data from an external system.

A. Because delta Lake versions all data and supports time travel, it is not possible for user error or malicious actors to permanently delete production data, as such it is generally safe to mount production data anywhere.

B. All developer, testing and production code and data should exist in a single unified workspace; creating separate environments for testing and development further reduces risks.

C. Because access to production data will always be verified using passthrough credentials it is safe to mount data to any Databricks development environment.

D. In environments where interactive code will be executed, production data should only be accessible with read permissions; creating isolated databases for each environment further reduces risks.

A. def transform_data(input_df):
# transformation logic here
return output_df
test_input = spark.createDataFrame([(1, " a " )], [ " id " , " value " ]) assertDataFrameEqual(transform_data(test_input), expected)

B. def upper_value(df):
return df.withColumn( " value_upper " , upper(col( " value " )))
def filter_positive(df):
return df.filter(df[ " id " ] > 0)
pipeline_df = df.transform(upper_value).transform(filter_positive)

C. def upper_transform(df):
return df.withColumn( " value_upper " , upper(col( " value " )))
actual = test_input.transform(upper_transform)
assertDataFrameEqual(actual, expected)

D. class Pipeline:
def transform(self, df):
return df.withColumn( " value_upper " , upper(col( " value " )))
pipeline = Pipeline()
assertDataFrameEqual(pipeline.transform(test_input), expected)

A. importlib.resource path

B. pypi.path

C. pylib.source

D. ,sys.path

E. os-path

A. Use a grouped_agg Pandas UDF that processes each stock symbol group independently, maintaining state through intermediate aggregation results that get passed between successive UDF calls via broadcast variables.

B. Use applyInPandas() on a Spark DataFrame that receives all rows for each stock symbol as a Pandas DataFrame, allowing processing within each group while maintaining state variables local to each group's processing function.

C. Use a SCALAR_ITER Pandas UDF with iterator-based processing, implementing state management through persistent storage (Delta tables) that gets updated after each batch to maintain continuity across iterator chunks.

D. Use a SCALAR Pandas UDF that processes the entire dataset at once, implementing custom partitioning logic within the UDF to group by stock symbol and maintain state using global variables shared across all executor processes.

A. Regex

B. Scala Datasets

C. Julia

D. pyspsark.ml.feature

E. C++

A. import dlt
@dlt.view
def raw_customers():
return spark.format.json( " s3://my-bucket/raw-customers/ " )

B. import dlt
@dlt.table
def raw_customers():
return spark.read.format( " parquet " ).load( " s3://my-bucket/raw-customers/ " )

C. import dlt
@dlt.table
def raw_customers():
return spark.read.format( " csv " ).load( " s3://my-bucket/raw-customers/ " )

D. import dlt
@dlt.table
def raw_customers():
return spark.read.json( " s3://my-bucket/raw-customers/ " )

A. Databricks has autotuned to a smaller target file size based on the amount of data in each partition

B. Databricks has autotuned to a smaller target file size based on the overall size of data in the table

C. Z-order indices calculated on the table are preventing file compaction C Bloom filler indices calculated on the table are preventing file compaction

D. Databricks has autotuned to a smaller target file size to reduce duration of MERGE operations

A. Write custom PySpark code using df.describe() and df.schema to programmatically generate basic statistical descriptions for each column.

B. Use the DESCRIBE TABLE command to extract existing schema information and manually write descriptions based on column names and data types.

C. Use Delta Lake's DESCRIBE HISTORY command to analyze table evolution and infer column purposes from historical changes.

D. Navigate to the table in Databricks Catalog Explorer, select the table schema view, and use the AI Generate option which leverages artificial intelligence to automatically create meaningful column descriptions based on column names, data types, sample values, and data patterns.

A. Use Repos to create a new branch commit all changes and push changes to the remote Git repertory.

B. Use Repos to merge all differences and make a pull request back to the remote repository.

C. Use repos to create a fork of the remote repository commit all changes and make a pull request on the source repository

D. Use repos to merge all difference and make a pull request back to the remote repository.

A. A logical table will persist the physical plan to the Hive Metastore in the Databricks control plane.

B. A managed table will be created in the DBFS root storage container.

C. An external table will be created in the storage container mounted to /mnt/finance eda bucket.

D. A logical table will persist the query plan to the Hive Metastore in the Databricks control plane.

E. An managed table will be created in the storage container mounted to /mnt/finance eda bucket.

A. Apply Z-ordering , since it allows flexible reorganization of data layout without rewriting existing files and adapts easily to new filter columns.

B. Enable liquid clustering , as it efficiently handles data skew, allows clustering keys to be changed without rewriting existing data, and adapts to evolving query patterns.

C. Use Hive-style partitioning , as it provides efficient data skipping and is easy to change partition columns at any time.

D. Combine partitioning and Z-ordering to maximize flexibility and minimize maintenance as query patterns change.

A. Open the Query Profiler associated with that query and use the Aggregated task time metric.

B. In the Spark UI, take the sum of all task durations that ran across all stages for all jobs associated with that query.

C. In the Spark UI, take the job duration of the longest-running job associated with that query.

D. Open the Query Profiler associated with that query and use the Total wall-clock duration metric.