pyspark.sql.functions.overlay#

pyspark.sql.functions.overlay(src, replace, pos, len=-1)[source]#

Overlay the specified portion of src with replace, starting from byte position pos of src and proceeding for len bytes.

Added in version 3.0.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

srcColumn or column name: the string that will be replaced. A column that evaluates to a string or binary.
replaceColumn or column name: the substitution string. A column that evaluates to a string or binary.
posColumn or column name or int: the starting position in src. A column that evaluates to an integer.
lenColumn or column name or int, optional: the number of bytes to replace in src string by ‘replace’ defaults to -1, which represents the length of the ‘replace’ string. A column that evaluates to an integer.

Returns:

Column: string with replaced values. Returns a column of the same type as the input.

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([("SPARK_SQL", "CORE")], ("x", "y"))
>>> df.select("*", sf.overlay("x", df.y, 7)).show()
+---------+----+--------------------+
|        x|   y|overlay(x, y, 7, -1)|
+---------+----+--------------------+
|SPARK_SQL|CORE|          SPARK_CORE|
+---------+----+--------------------+

>>> df.select("*", sf.overlay("x", df.y, 7, 0)).show()
+---------+----+-------------------+
|        x|   y|overlay(x, y, 7, 0)|
+---------+----+-------------------+
|SPARK_SQL|CORE|      SPARK_CORESQL|
+---------+----+-------------------+

>>> df.select("*", sf.overlay("x", "y", 7, 2)).show()
+---------+----+-------------------+
|        x|   y|overlay(x, y, 7, 2)|
+---------+----+-------------------+
|SPARK_SQL|CORE|        SPARK_COREL|
+---------+----+-------------------+