An upstream system is emitting change data capture (CDC) logs that are being written to a cloud object storage directory. Each record in the log indicates the change type (insert, update, or delete) and the values for each field after the change. The source table has a primary key identified by the field pk_id.
For auditing purposes, the data governance team wishes to maintain a full record of all values that have ever been valid in the source system. For analytical purposes, only the most recent value for each record needs to be recorded. The Databricks job to ingest these records occurs once per hour, but each individual record may have changed multiple times over the course of an hour.
Which solution meets these requirements?

• A. Create a separate history table for each pk_id resolve the current state of the table by running a union all filtering the history tables for the most recent state.
• B. Use merge into to insert, update, or delete the most recent entry for each pk_id into a bronze table, then propagate all changes throughout the system.
• C. Iterate through an ordered set of changes to the table, applying each in turn; rely on Delta Lake's versioning ability to create an audit log.
• D. Ingest all log information into a bronze table; use merge into to insert, update, or delete the most recent entry for each pk_id into a silver table to recreate the current table state.
• E. Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.

Explanation: Only visible for Actual4Exams members. You can sign-up / login (it's free).

The downstream consumers of a Delta Lake table have been complaining about data quality issues impacting performance in their applications. Specifically, they have complained that invalid latitude and longitude values in the activity_details table have been breaking their ability to use other geolocation processes.
A junior engineer has written the following code to add CHECK constraints to the Delta Lake table:

A senior engineer has confirmed the above logic is correct and the valid ranges for latitude and longitude are provided, but the code fails when executed.
Which statement explains the cause of this failure?

• A. The activity details table already contains records; CHECK constraints can only be added prior to inserting values into a table.
• B. The activity details table already exists; CHECK constraints can only be added during initial table creation.
• C. The current table schema does not contain the field valid coordinates; schema evolution will need to be enabled before altering the table to add a constraint.
• D. Because another team uses this table to support a frequently running application, two-phase locking is preventing the operation from committing.
• E. The activity details table already contains records that violate the constraints; all existing data must pass CHECK constraints in order to add them to an existing table.

Explanation: Only visible for Actual4Exams members. You can sign-up / login (it's free).

A distributed team of data analysts share computing resources on an interactive cluster with autoscaling configured. In order to better manage costs and query throughput, the workspace administrator is hoping to evaluate whether cluster upscaling is caused by many concurrent users or resource-intensive queries.
In which location can one review the timeline for cluster resizing events?

• A. Driver's log file
• B. Workspace audit logs
• C. Cluster Event Log
• D. Ganglia
• E. Executor's log file

Which statement describes Delta Lake optimized writes?

• A. A shuffle occurs prior to writing to try to group data together resulting in fewer files instead of each executor writing multiple files based on directory partitions.
• B. An asynchronous job runs after the write completes to detect if files could be further compacted; yes, an OPTIMIZE job is executed toward a default of 1 GB.
• C. Optimized writes logical partitions instead of directory partitions partition boundaries are only represented in metadata fewer small files are written.
• D. Before a job cluster terminates, OPTIMIZE is executed on all tables modified during the most recent job.

Explanation: Only visible for Actual4Exams members. You can sign-up / login (it's free).

A data architect has designed a system in which two Structured Streaming jobs will concurrently write to a single bronze Delta table. Each job is subscribing to a different topic from an Apache Kafka source, but they will write data with the same schema. To keep the directory structure simple, a data engineer has decided to nest a checkpoint directory to be shared by both streams.
The proposed directory structure is displayed below:

Which statement describes whether this checkpoint directory structure is valid for the given scenario and why?

• A. Yes; Delta Lake supports infinite concurrent writers.
• B. No; only one stream can write to a Delta Lake table.
• C. Yes; both of the streams can share a single checkpoint directory.
• D. No; each of the streams needs to have its own checkpoint directory.
• E. No; Delta Lake manages streaming checkpoints in the transaction log.

Explanation: Only visible for Actual4Exams members. You can sign-up / login (it's free).