Clear your concepts with Associate-Developer-Apache-Spark Questions Before Attempting Real exam [Q64-Q81]

Clear your concepts with Associate-Developer-Apache-Spark Questions Before Attempting Real exam

Get professional help from our Associate-Developer-Apache-Spark Dumps PDF

A. 1. length
2. regexp_replace
3. lower
4. col("itemName")
5. alias
B. 1. lower
2. regexp_replace
3. length
4. "itemName"
5. alias
C. 1. size
2. regexp_replace
3. lower
4. "itemName"
5. alias
D. 1. size
2. regexp_extract
3. lower
4. col("itemName")
5. alias
E. 1. length
2. regexp_extract
3. upper
4. col("itemName")
5. as

Answer: A

Explanation:
Explanation
Correct code block:
itemsDf.select(length(regexp_replace(lower(col("itemName")), "a|e|i|o|u|\s", "")).alias("consonant_ct")) Returned DataFrame:
+------------+
|consonant_ct|
+------------+
| 19|
| 16|
| 10|
+------------+
This question tries to make you think about the string functions Spark provides and in which order they should be applied. Arguably the most difficult part, the regular expression "a|e|i|o|u|
\s", is not a numbered blank. However, if you are not familiar with the string functions, it may be a good idea to review those before the exam.
The size operator and the length operator can easily be confused. size works on arrays, while length works on strings. Luckily, this is something you can read up about in the documentation.
The code block works by first converting all uppercase letters in column itemName into lowercase (the lower() part). Then, it replaces all vowels by "nothing" - an empty character "" (the regexp_replace() part). Now, only lowercase characters without spaces are included in the DataFrame. Then, per row, the length operator counts these remaining characters. Note that column itemName in itemsDf does not include any numbers or other characters, so we do not need to make any provisions for these. Finally, by using the alias() operator, we rename the resulting column to consonant_ct.
More info:
- lower: pyspark.sql.functions.lower - PySpark 3.1.2 documentation
- regexp_replace: pyspark.sql.functions.regexp_replace - PySpark 3.1.2 documentation
- length: pyspark.sql.functions.length - PySpark 3.1.2 documentation
- alias: pyspark.sql.Column.alias - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 2

NEW QUESTION 65
The code block shown below should return an exact copy of DataFrame transactionsDf that does not include rows in which values in column storeId have the value 25. Choose the answer that correctly fills the blanks in the code block to accomplish this.

A. transactionsDf.where(transactionsDf.storeId!=25)
B. transactionsDf.drop(transactionsDf.storeId==25)
C. transactionsDf.filter(transactionsDf.storeId==25)
D. transactionsDf.select(transactionsDf.storeId!=25)
E. transactionsDf.remove(transactionsDf.storeId==25)

Answer: A

Explanation:
Explanation
transactionsDf.where(transactionsDf.storeId!=25)
Correct. DataFrame.where() is an alias for the DataFrame.filter() method. Using this method, it is straightforward to filter out rows that do not have value 25 in column storeId.
transactionsDf.select(transactionsDf.storeId!=25)
Wrong. The select operator allows you to build DataFrames column-wise, but when using it as shown, it does not filter out rows.
transactionsDf.filter(transactionsDf.storeId==25)
Incorrect. Although the filter expression works for filtering rows, the == in the filtering condition is inappropriate. It should be != instead.
transactionsDf.drop(transactionsDf.storeId==25)
No. DataFrame.drop() is used to remove specific columns, but not rows, from the DataFrame.
transactionsDf.remove(transactionsDf.storeId==25)
False. There is no DataFrame.remove() operator in PySpark.
More info: pyspark.sql.DataFrame.where - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3

NEW QUESTION 66
The code block shown below should return a column that indicates through boolean variables whether rows in DataFrame transactionsDf have values greater or equal to 20 and smaller or equal to
30 in column storeId and have the value 2 in column productId. Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__((__2__.__3__) __4__ (__5__))

A. 1. select
2. col("storeId")
3. between(20, 30)
4. and
5. col("productId")==2
B. 1. select
2. col("storeId")
3. between(20, 30)
4. &&
5. col("productId")=2
C. 1. select
2. "storeId"
3. between(20, 30)
4. &&
5. col("productId")==2
D. 1. select
2. col("storeId")
3. between(20, 30)
4. &
5. col("productId")==2
E. 1. where
2. col("storeId")
3. geq(20).leq(30)
4. &
5. col("productId")==2

Answer: B

Explanation:
Explanation
Correct code block:
transactionsDf.select((col("storeId").between(20, 30)) & (col("productId")==2)) Although this question may make you think that it asks for a filter or where statement, it does not. It asks explicity to return a column with booleans - this should point you to the select statement.
Another trick here is the rarely used between() method. It exists and resolves to ((storeId >= 20) AND (storeId
<= 30)) in SQL. geq() and leq() do not exist.
Another riddle here is how to chain the two conditions. The only valid answer here is &. Operators like && or and are not valid. Other boolean operators that would be valid in Spark are | and.
Static notebook | Dynamic notebook: See test 1

NEW QUESTION 67
Which of the following describes a valid concern about partitioning?

A. A shuffle operation returns 200 partitions if not explicitly set.
B. The coalesce() method should be used to increase the number of partitions.
C. Short partition processing times are indicative of low skew.
D. No data is exchanged between executors when coalesce() is run.
E. Decreasing the number of partitions reduces the overall runtime of narrow transformations if there are more executors available than partitions.

Answer: A

Explanation:
Explanation
A shuffle operation returns 200 partitions if not explicitly set.
Correct. 200 is the default value for the Spark property spark.sql.shuffle.partitions. This property determines how many partitions Spark uses when shuffling data for joins or aggregations.
The coalesce() method should be used to increase the number of partitions.
Incorrect. The coalesce() method can only be used to decrease the number of partitions.
Decreasing the number of partitions reduces the overall runtime of narrow transformations if there are more executors available than partitions.
No. For narrow transformations, fewer partitions usually result in a longer overall runtime, if more executors are available than partitions.
A narrow transformation does not include a shuffle, so no data need to be exchanged between executors.
Shuffles are expensive and can be a bottleneck for executing Spark workloads.
Narrow transformations, however, are executed on a per-partition basis, blocking one executor per partition.
So, it matters how many executors are available to perform work in parallel relative to the number of partitions. If the number of executors is greater than the number of partitions, then some executors are idle while other process the partitions. On the flip side, if the number of executors is smaller than the number of partitions, the entire operation can only be finished after some executors have processed multiple partitions, one after the other. To minimize the overall runtime, one would want to have the number of partitions equal to the number of executors (but not more).
So, for the scenario at hand, increasing the number of partitions reduces the overall runtime of narrow transformations if there are more executors available than partitions.
No data is exchanged between executors when coalesce() is run.
No. While coalesce() avoids a full shuffle, it may still cause a partial shuffle, resulting in data exchange between executors.
Short partition processing times are indicative of low skew.
Incorrect. Data skew means that data is distributed unevenly over the partitions of a dataset. Low skew therefore means that data is distributed evenly.
Partition processing time, the time that executors take to process partitions, can be indicative of skew if some executors take a long time to process a partition, but others do not. However, a short processing time is not per se indicative a low skew: It may simply be short because the partition is small.
A situation indicative of low skew may be when all executors finish processing their partitions in the same timeframe. High skew may be indicated by some executors taking much longer to finish their partitions than others. But the answer does not make any comparison - so by itself it does not provide enough information to make any assessment about skew.
More info: Spark Repartition & Coalesce - Explained and Performance Tuning - Spark 3.1.2 Documentation

NEW QUESTION 68
The code block shown below should return a DataFrame with two columns, itemId and col. In this DataFrame, for each element in column attributes of DataFrame itemDf there should be a separate row in which the column itemId contains the associated itemId from DataFrame itemsDf. The new DataFrame should only contain rows for rows in DataFrame itemsDf in which the column attributes contains the element cozy.
A sample of DataFrame itemsDf is below.
Code block:
itemsDf.__1__(__2__).__3__(__4__, __5__(__6__))

A. 1. filter
2. "array_contains(attributes, 'cozy')"
3. select
4. "itemId"
5. explode
6. "attributes"
B. 1. filter
2. array_contains("cozy")
3. select
4. "itemId"
5. explode
6. "attributes"
C. 1. where
2. "array_contains(attributes, 'cozy')"
3. select
4. itemId
5. explode
6. attributes
D. 1. filter
2. "array_contains(attributes, cozy)"
3. select
4. "itemId"
5. explode
6. "attributes"
E. 1. filter
2. "array_contains(attributes, 'cozy')"
3. select
4. "itemId"
5. map
6. "attributes"

Answer: A

Explanation:
Explanation
The correct code block is:
itemsDf.filter("array_contains(attributes, 'cozy')").select("itemId", explode("attributes")) The key here is understanding how to use array_contains(). You can either use it as an expression in a string, or you can import it from pyspark.sql.functions. In that case, the following would also work:
itemsDf.filter(array_contains("attributes", "cozy")).select("itemId", explode("attributes")) Static notebook | Dynamic notebook: See test 1 (https://flrs.github.io/spark_practice_tests_code/#1/29.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

NEW QUESTION 69
Which of the following code blocks returns a DataFrame that matches the multi-column DataFrame itemsDf, except that integer column itemId has been converted into a string column?

A. itemsDf.select(cast("itemId", "string"))
B. itemsDf.withColumn("itemId", col("itemId").cast("string"))
(Correct)
C. itemsDf.withColumn("itemId", convert("itemId", "string"))
D. itemsDf.withColumn("itemId", col("itemId").convert("string"))
E. spark.cast(itemsDf, "itemId", "string")

Answer: B

Explanation:
Explanation
itemsDf.withColumn("itemId", col("itemId").cast("string"))
Correct. You can convert the data type of a column using the cast method of the Column class. Also note that you will have to use the withColumn method on itemsDf for replacing the existing itemId column with the new version that contains strings.
itemsDf.withColumn("itemId", col("itemId").convert("string"))
Incorrect. The Column object that col("itemId") returns does not have a convert method.
itemsDf.withColumn("itemId", convert("itemId", "string"))
Wrong. Spark's spark.sql.functions module does not have a convert method. The question is trying to mislead you by using the word "converted". Type conversion is also called "type casting". This may help you remember to look for a cast method instead of a convert method (see correct answer).
itemsDf.select(astype("itemId", "string"))
False. While astype is a method of Column (and an alias of Column.cast), it is not a method of pyspark.sql.functions (what the code block implies). In addition, the question asks to return a full DataFrame that matches the multi-column DataFrame itemsDf. Selecting just one column from itemsDf as in the code block would just return a single-column DataFrame.
spark.cast(itemsDf, "itemId", "string")
No, the Spark session (called by spark) does not have a cast method. You can find a list of all methods available for the Spark session linked in the documentation below.
More info:
- pyspark.sql.Column.cast - PySpark 3.1.2 documentation
- pyspark.sql.Column.astype - PySpark 3.1.2 documentation
- pyspark.sql.SparkSession - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3

NEW QUESTION 70
The code block shown below should show information about the data type that column storeId of DataFrame transactionsDf contains. Choose the answer that correctly fills the blanks in the code block to accomplish this.
Code block:
transactionsDf.__1__(__2__).__3__

A. 1. select
2. "storeId"
3. printSchema()
B. 1. limit
2. 1
3. columns
C. 1. select
2. "storeId"
3. print_schema()
D. 1. limit
2. "storeId"
3. printSchema()
E. 1. select
2. storeId
3. dtypes

Answer: B

Explanation:
Explanation
Correct code block:
transactionsDf.select("storeId").printSchema()
The difficulty of this question is that it is hard to solve with the stepwise first-to-last-gap approach that has worked well for similar questions, since the answer options are so different from one another. Instead, you might want to eliminate answers by looking for patterns of frequently wrong answers.
A first pattern that you may recognize by now is that column names are not expressed in quotes. For this reason, the answer that includes storeId should be eliminated.
By now, you may have understood that the DataFrame.limit() is useful for returning a specified amount of rows. It has nothing to do with specific columns. For this reason, the answer that resolves to limit("storeId") can be eliminated.
Given that we are interested in information about the data type, you should question whether the answer that resolves to limit(1).columns provides you with this information. While DataFrame.columns is a valid call, it will only report back column names, but not column types. So, you can eliminate this option.
The two remaining options either use the printSchema() or print_schema() command. You may remember that DataFrame.printSchema() is the only valid command of the two. The select("storeId") part just returns the storeId column of transactionsDf - this works here, since we are only interested in that column's type anyways.
More info: pyspark.sql.DataFrame.printSchema - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 3

NEW QUESTION 71
Which of the following code blocks creates a new 6-column DataFrame by appending the rows of the
6-column DataFrame yesterdayTransactionsDf to the rows of the 6-column DataFrame todayTransactionsDf, ignoring that both DataFrames have different column names?

A. union(todayTransactionsDf, yesterdayTransactionsDf)
B. todayTransactionsDf.concat(yesterdayTransactionsDf)
C. todayTransactionsDf.unionByName(yesterdayTransactionsDf, allowMissingColumns=True)
D. todayTransactionsDf.unionByName(yesterdayTransactionsDf)
E. todayTransactionsDf.union(yesterdayTransactionsDf)

Answer: E

Explanation:
Explanation
todayTransactionsDf.union(yesterdayTransactionsDf)
Correct. The union command appends rows of yesterdayTransactionsDf to the rows of todayTransactionsDf, ignoring that both DataFrames have different column names. The resulting DataFrame will have the column names of DataFrame todayTransactionsDf.
todayTransactionsDf.unionByName(yesterdayTransactionsDf)
No. unionByName specifically tries to match columns in the two DataFrames by name and only appends values in columns with identical names across the two DataFrames. In the form presented above, the command is a great fit for joining DataFrames that have exactly the same columns, but in a different order. In this case though, the command will fail because the two DataFrames have different columns.
todayTransactionsDf.unionByName(yesterdayTransactionsDf, allowMissingColumns=True) No. The unionByName command is described in the previous explanation. However, with the allowMissingColumns argument set to True, it is no longer an issue that the two DataFrames have different column names. Any columns that do not have a match in the other DataFrame will be filled with null where there is no value. In the case at hand, the resulting DataFrame will have 7 or more columns though, so it this command is not the right answer.
union(todayTransactionsDf, yesterdayTransactionsDf)
No, there is no union method in pyspark.sql.functions.
todayTransactionsDf.concat(yesterdayTransactionsDf)
Wrong, the DataFrame class does not have a concat method.
More info: pyspark.sql.DataFrame.union - PySpark 3.1.2 documentation,
pyspark.sql.DataFrame.unionByName - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3

NEW QUESTION 72
Which of the following describes the role of tasks in the Spark execution hierarchy?

A. Tasks are the second-smallest element in the execution hierarchy.
B. Within one task, the slots are the unit of work done for each partition of the data.
C. Tasks with wide dependencies can be grouped into one stage.
D. Stages with narrow dependencies can be grouped into one task.
E. Tasks are the smallest element in the execution hierarchy.

Answer: E

Explanation:
Explanation
Stages with narrow dependencies can be grouped into one task.
Wrong, tasks with narrow dependencies can be grouped into one stage.
Tasks with wide dependencies can be grouped into one stage.
Wrong, since a wide transformation causes a shuffle which always marks the boundary of a stage. So, you cannot bundle multiple tasks that have wide dependencies into a stage.
Tasks are the second-smallest element in the execution hierarchy.
No, they are the smallest element in the execution hierarchy.
Within one task, the slots are the unit of work done for each partition of the data.
No, tasks are the unit of work done per partition. Slots help Spark parallelize work. An executor can have multiple slots which enable it to process multiple tasks in parallel.

NEW QUESTION 73
Which of the following code blocks silently writes DataFrame itemsDf in avro format to location fileLocation if a file does not yet exist at that location?

A. spark.DataFrameWriter(itemsDf).format("avro").write(fileLocation)
B. itemsDf.write.format("avro").mode("errorifexists").save(fileLocation)
C. itemsDf.save.format("avro").mode("ignore").write(fileLocation)
D. itemsDf.write.format("avro").mode("ignore").save(fileLocation)
E. itemsDf.write.avro(fileLocation)

Answer: E

Explanation:
Explanation
The trick in this question is knowing the "modes" of the DataFrameWriter. Mode ignore will ignore if a file already exists and not replace that file, but also not throw an error. Mode errorifexists will throw an error, and is the default mode of the DataFrameWriter. The question NO:
explicitly calls for the DataFrame to be "silently" written if it does not exist, so you need to specify mode("ignore") here to avoid having Spark communicate any error to you if the file already exists.
The `overwrite' mode would not be right here, since, although it would be silent, it would overwrite the already-existing file. This is not what the question asks for.
It is worth noting that the option starting with spark.DataFrameWriter(itemsDf) cannot work, since spark references the SparkSession object, but that object does not provide the DataFrameWriter.
As you can see in the documentation (below), DataFrameWriter is part of PySpark's SQL API, but not of its SparkSession API.
More info:
DataFrameWriter: pyspark.sql.DataFrameWriter.save - PySpark 3.1.1 documentation SparkSession API: Spark SQL - PySpark 3.1.1 documentation Static notebook | Dynamic notebook: See test 1

NEW QUESTION 74
The code block shown below should convert up to 5 rows in DataFrame transactionsDf that have the value 25 in column storeId into a Python list. Choose the answer that correctly fills the blanks in the code block to accomplish this.
Code block:
transactionsDf.__1__(__2__).__3__(__4__)

A. 1. filter
2. "storeId"==25
3. collect
4. 5
B. 1. select
2. storeId==25
3. head
4. 5
C. 1. filter
2. col("storeId")==25
3. collect
4. 5
D. 1. filter
2. col("storeId")==25
3. toLocalIterator
4. 5
E. 1. filter
2. col("storeId")==25
3. take
4. 5

Answer: E

Explanation:
Explanation
The correct code block is:
transactionsDf.filter(col("storeId")==25).take(5)
Any of the options with collect will not work because collect does not take any arguments, and in both cases the argument 5 is given.
The option with toLocalIterator will not work because the only argument to toLocalIterator is prefetchPartitions which is a boolean, so passing 5 here does not make sense.
The option using head will not work because the expression passed to select is not proper syntax. It would work if the expression would be col("storeId")==25.
Static notebook | Dynamic notebook: See test 1
(https://flrs.github.io/spark_practice_tests_code/#1/24.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

NEW QUESTION 75
The code block displayed below contains an error. The code block should return the average of rows in column value grouped by unique storeId. Find the error.
Code block:
transactionsDf.agg("storeId").avg("value")

A. Instead of avg("value"), avg(col("value")) should be used.
B. The avg("value") should be specified as a second argument to agg() instead of being appended to it.
C. All column names should be wrapped in col() operators.
D. "storeId" and "value" should be swapped.
E. agg should be replaced by groupBy.

Answer: E

Explanation:
Explanation
Static notebook | Dynamic notebook: See test 1
(https://flrs.github.io/spark_practice_tests_code/#1/30.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

NEW QUESTION 76
Which of the following code blocks applies the boolean-returning Python function evaluateTestSuccess to column storeId of DataFrame transactionsDf as a user-defined function?

A. 1.evaluateTestSuccessUDF = udf(evaluateTestSuccess)
2.transactionsDf.withColumn("result", evaluateTestSuccessUDF(col("storeId")))
B. 1.from pyspark.sql import types as T
2.evaluateTestSuccessUDF = udf(evaluateTestSuccess, T.BooleanType())
3.transactionsDf.withColumn("result", evaluateTestSuccessUDF(col("storeId")))
C. 1.evaluateTestSuccessUDF = udf(evaluateTestSuccess)
2.transactionsDf.withColumn("result", evaluateTestSuccessUDF(storeId))
D. 1.from pyspark.sql import types as T
2.evaluateTestSuccessUDF = udf(evaluateTestSuccess, T.BooleanType())
3.transactionsDf.withColumn("result", evaluateTestSuccess(col("storeId")))
E. 1.from pyspark.sql import types as T
2.evaluateTestSuccessUDF = udf(evaluateTestSuccess, T.IntegerType())
3.transactionsDf.withColumn("result", evaluateTestSuccess(col("storeId")))

Answer: B

Explanation:
Explanation
Recognizing that the UDF specification requires a return type (unless it is a string, which is the default) is important for solving this question. In addition, you should make sure that the generated UDF (evaluateTestSuccessUDF) and not the Python function (evaluateTestSuccess) is applied to column storeId.
More info: pyspark.sql.functions.udf - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 2

NEW QUESTION 77
The code block shown below should store DataFrame transactionsDf on two different executors, utilizing the executors' memory as much as possible, but not writing anything to disk. Choose the answer that correctly fills the blanks in the code block to accomplish this.
1.from pyspark import StorageLevel
2.transactionsDf.__1__(StorageLevel.__2__).__3__

A. 1. persist
2. MEMORY_ONLY_2
3. select()
B. 1. cache
2. DISK_ONLY_2
3. count()
C. 1. persist
2. MEMORY_ONLY_2
3. count()
D. 1. persist
2. DISK_ONLY_2
3. count()
E. 1. cache
2. MEMORY_ONLY_2
3. count()

Answer: C

Explanation:
Explanation
Correct code block:
from pyspark import StorageLevel
transactionsDf.persist(StorageLevel.MEMORY_ONLY_2).count()
Only persist takes different storage levels, so any option using cache() cannot be correct. persist() is evaluated lazily, so an action needs to follow this command. select() is not an action, but count() is - so all options using select() are incorrect.
Finally, the question states that "the executors' memory should be utilized as much as possible, but not writing anything to disk". This points to a MEMORY_ONLY storage level. In this storage level, partitions that do not fit into memory will be recomputed when they are needed, instead of being written to disk, as with the storage option MEMORY_AND_DISK. Since the data need to be duplicated across two executors, _2 needs to be appended to the storage level.
Static notebook | Dynamic notebook: See test 2

NEW QUESTION 78
Which of the following code blocks returns a new DataFrame with only columns predError and values of every second row of DataFrame transactionsDf?
Entire DataFrame transactionsDf:
1.+-------------+---------+-----+-------+---------+----+
2.|transactionId|predError|value|storeId|productId| f|
3.+-------------+---------+-----+-------+---------+----+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.| 4| null| null| 3| 2|null|
8.| 5| null| null| null| 2|null|
9.| 6| 3| 2| 25| 2|null|
10.+-------------+---------+-----+-------+---------+----+

A. transactionsDf.filter(col("transactionId").isin([3,4,6])).select([predError, value])
B. transactionsDf.filter("transactionId" % 2 == 0).select("predError", "value")
C. transactionsDf.filter(col(transactionId).isin([3,4,6]))
D. 1.transactionsDf.createOrReplaceTempView("transactionsDf")
2.spark.sql("FROM transactionsDf SELECT predError, value WHERE transactionId % 2 = 2")
E. transactionsDf.filter(col("transactionId") % 2 == 0).select("predError", "value") (Correct)
F. transactionsDf.select(col("transactionId").isin([3,4,6]), "predError", "value")

Answer: E

Explanation:
Explanation
Output of correct code block:
+---------+-----+
|predError|value|
+---------+-----+
| 6| 7|
| null| null|
| 3| 2|
+---------+-----+
This is not an easy question to solve. You need to know that % stands for the module operator in Python. % 2 will return true for every second row. The statement using spark.sql gets it almost right (the modulo operator exists in SQL as well), but % 2 = 2 will never yield true, since modulo 2 is either 0 or 1.
Other answers are wrong since they are missing quotes around the column names and/or use filter or select incorrectly.
If you have any doubts about SparkSQL and answer options 3 and 4 in this question, check out the notebook I created as a response to a related student question.
Static notebook | Dynamic notebook: See test 1

NEW QUESTION 79
Which of the following is a characteristic of the cluster manager?

A. In client mode, the cluster manager runs on the edge node.
B. The cluster manager transforms jobs into DAGs.
C. The cluster manager receives input from the driver through the SparkContext.
D. Each cluster manager works on a single partition of data.
E. The cluster manager does not exist in standalone mode.

Answer: C

Explanation:
Explanation
The cluster manager receives input from the driver through the SparkContext.
Correct. In order for the driver to contact the cluster manager, the driver launches a SparkContext. The driver then asks the cluster manager for resources to launch executors.
In client mode, the cluster manager runs on the edge node.
No. In client mode, the cluster manager is independent of the edge node and runs in the cluster.
The cluster manager does not exist in standalone mode.
Wrong, the cluster manager exists even in standalone mode. Remember, standalone mode is an easy means to deploy Spark across a whole cluster, with some limitations. For example, in standalone mode, no other frameworks can run in parallel with Spark. The cluster manager is part of Spark in standalone deployments however and helps launch and maintain resources across the cluster.
The cluster manager transforms jobs into DAGs.
No, transforming jobs into DAGs is the task of the Spark driver.
Each cluster manager works on a single partition of data.
No. Cluster managers do not work on partitions directly. Their job is to coordinate cluster resources so that they can be requested by and allocated to Spark drivers.
More info: Introduction to Core Spark Concepts * BigData

NEW QUESTION 80
Which of the following is the deepest level in Spark's execution hierarchy?

A. Executor
B. Stage
C. Task
D. Slot
E. Job

Answer: C

Explanation:
Explanation
The hierarchy is, from top to bottom: Job, Stage, Task.
Executors and slots facilitate the execution of tasks, but they are not directly part of the hierarchy. Executors are launched by the driver on worker nodes for the purpose of running a specific Spark application. Slots help Spark parallelize work. An executor can have multiple slots which enable it to process multiple tasks in parallel.

NEW QUESTION 81
......

Achieve the Associate-Developer-Apache-Spark Exam Best Results with Help from Databricks Certified Experts: https://www.testpassed.com/Associate-Developer-Apache-Spark-still-valid-exam.html

Give You Free Regular Updates on Associate-Developer-Apache-Spark Exam Questions: https://drive.google.com/open?id=1PUmjLcPKZ9f8SsTsmpYInepwcVXygXWs

Clear your concepts with Associate-Developer-Apache-Spark Questions Before Attempting Real exam [Q64-Q81]

Related Articles

Latest Test Passed

Useful Links

Contact Us