R: R Front End for 'Apache Spark'

# R Front End for 'Apache Spark'

## Documentation for package ‘SparkR’ version 3.1.2

- DESCRIPTION file.
- User guides, package vignettes and other documentation.

## Help Pages

A B C D E F G H I J K L M N O P Q R S T U V W X Y misc

## -- A --

abs | Math functions for Column operations
abs-method | Math functions for Column operations
acos | Math functions for Column operations
acos-method | Math functions for Column operations
acosh | Math functions for Column operations
acosh-method | Math functions for Column operations
add_months | Date time arithmetic functions for Column operations
add_months-method | Date time arithmetic functions for Column operations
AFTSurvivalRegressionModel-class | S4 class that represents a AFTSurvivalRegressionModel
agg | summarize
agg-method | summarize
alias | alias
alias-method | alias
ALSModel-class | S4 class that represents an ALSModel
approxCountDistinct | Aggregate functions for Column operations
approxCountDistinct-method | Aggregate functions for Column operations
approxQuantile | Calculates the approximate quantiles of numerical columns of a SparkDataFrame
approxQuantile-method | Calculates the approximate quantiles of numerical columns of a SparkDataFrame
approx_count_distinct | Aggregate functions for Column operations
approx_count_distinct-method | Aggregate functions for Column operations
arrange | Arrange Rows by Variables
arrange-method | Arrange Rows by Variables
arrays_overlap | Collection functions for Column operations
arrays_overlap-method | Collection functions for Column operations
arrays_zip | Collection functions for Column operations
arrays_zip-method | Collection functions for Column operations
arrays_zip_with | Collection functions for Column operations
arrays_zip_with-method | Collection functions for Column operations
array_aggregate | Collection functions for Column operations
array_aggregate-method | Collection functions for Column operations
array_contains | Collection functions for Column operations
array_contains-method | Collection functions for Column operations
array_distinct | Collection functions for Column operations
array_distinct-method | Collection functions for Column operations
array_except | Collection functions for Column operations
array_except-method | Collection functions for Column operations
array_exists | Collection functions for Column operations
array_exists-method | Collection functions for Column operations
array_filter | Collection functions for Column operations
array_filter-method | Collection functions for Column operations
array_forall | Collection functions for Column operations
array_forall-method | Collection functions for Column operations
array_intersect | Collection functions for Column operations
array_intersect-method | Collection functions for Column operations
array_join | Collection functions for Column operations
array_join-method | Collection functions for Column operations
array_max | Collection functions for Column operations
array_max-method | Collection functions for Column operations
array_min | Collection functions for Column operations
array_min-method | Collection functions for Column operations
array_position | Collection functions for Column operations
array_position-method | Collection functions for Column operations
array_remove | Collection functions for Column operations
array_remove-method | Collection functions for Column operations
array_repeat | Collection functions for Column operations
array_repeat-method | Collection functions for Column operations
array_sort | Collection functions for Column operations
array_sort-method | Collection functions for Column operations
array_to_vector | ML functions for Column operations
array_to_vector-method | ML functions for Column operations
array_transform | Collection functions for Column operations
array_transform-method | Collection functions for Column operations
array_union | Collection functions for Column operations
array_union-method | Collection functions for Column operations
as.data.frame | Download data from a SparkDataFrame into a R data.frame
as.data.frame-method | Download data from a SparkDataFrame into a R data.frame
as.DataFrame | Create a SparkDataFrame
asc | A set of operations working with SparkDataFrame columns
ascii | String functions for Column operations
ascii-method | String functions for Column operations
asc_nulls_first | A set of operations working with SparkDataFrame columns
asc_nulls_last | A set of operations working with SparkDataFrame columns
asin | Math functions for Column operations
asin-method | Math functions for Column operations
asinh | Math functions for Column operations
asinh-method | Math functions for Column operations
assert_true | Miscellaneous functions for Column operations
assert_true-method | Miscellaneous functions for Column operations
associationRules-method | FP-growth
atan | Math functions for Column operations
atan-method | Math functions for Column operations
atan2 | Math functions for Column operations
atan2-method | Math functions for Column operations
atanh | Math functions for Column operations
atanh-method | Math functions for Column operations
attach | Attach SparkDataFrame to R search path
attach-method | Attach SparkDataFrame to R search path
avg | avg
avg-method | avg
awaitTermination | awaitTermination
awaitTermination-method | awaitTermination

## -- B --

base64 | String functions for Column operations
base64-method | String functions for Column operations
between | between
between-method | between
bin | Math functions for Column operations
bin-method | Math functions for Column operations
BisectingKMeansModel-class | S4 class that represents a BisectingKMeansModel
bitwiseNOT | Non-aggregate functions for Column operations
bitwiseNOT-method | Non-aggregate functions for Column operations
broadcast | broadcast
broadcast-method | broadcast
bround | Math functions for Column operations
bround-method | Math functions for Column operations

## -- C --

cache | Cache
cache-method | Cache
cacheTable | Cache Table
cancelJobGroup | Cancel active jobs for the specified group
cast | Casts the column to a different data type.
cast-method | Casts the column to a different data type.
cbrt | Math functions for Column operations
cbrt-method | Math functions for Column operations
ceil | Math functions for Column operations
ceil-method | Math functions for Column operations
ceiling | Math functions for Column operations
ceiling-method | Math functions for Column operations
checkpoint | checkpoint
checkpoint-method | checkpoint
clearCache | Clear Cache
clearJobGroup | Clear current job group ID and its description
coalesce | Coalesce
coalesce-method | Coalesce
coalesce-method | Non-aggregate functions for Column operations
collect | Collects all the elements of a SparkDataFrame and coerces them into an R data.frame.
collect-method | Collects all the elements of a SparkDataFrame and coerces them into an R data.frame.
collect_list | Aggregate functions for Column operations
collect_list-method | Aggregate functions for Column operations
collect_set | Aggregate functions for Column operations
collect_set-method | Aggregate functions for Column operations
colnames | Column Names of SparkDataFrame
colnames-method | Column Names of SparkDataFrame
colnames<- | Column Names of SparkDataFrame
colnames<--method | Column Names of SparkDataFrame
coltypes | coltypes
coltypes-method | coltypes
coltypes<- | coltypes
coltypes<--method | coltypes
column | S4 class that represents a SparkDataFrame column
Column-class | S4 class that represents a SparkDataFrame column
column-method | S4 class that represents a SparkDataFrame column
columnfunctions | A set of operations working with SparkDataFrame columns
columns | Column Names of SparkDataFrame
columns-method | Column Names of SparkDataFrame
column_aggregate_functions | Aggregate functions for Column operations
column_avro_functions | Avro processing functions for Column operations
column_collection_functions | Collection functions for Column operations
column_datetime_diff_functions | Date time arithmetic functions for Column operations
column_datetime_functions | Date time functions for Column operations
column_math_functions | Math functions for Column operations
column_misc_functions | Miscellaneous functions for Column operations
column_ml_functions | ML functions for Column operations
column_nonaggregate_functions | Non-aggregate functions for Column operations
column_string_functions | String functions for Column operations
column_window_functions | Window functions for Column operations
concat | Collection functions for Column operations
concat-method | Collection functions for Column operations
concat_ws | String functions for Column operations
concat_ws-method | String functions for Column operations
contains | A set of operations working with SparkDataFrame columns
conv | Math functions for Column operations
conv-method | Math functions for Column operations
corr | corr
corr-method | corr
cos | Math functions for Column operations
cos-method | Math functions for Column operations
cosh | Math functions for Column operations
cosh-method | Math functions for Column operations
count | Count
count-method | Count
count-method | Returns the number of rows in a SparkDataFrame
countDistinct | Aggregate functions for Column operations
countDistinct-method | Aggregate functions for Column operations
cov | cov
cov-method | cov
covar_pop | cov
covar_pop-method | cov
covar_samp | cov
covar_samp-method | cov
crc32 | Miscellaneous functions for Column operations
crc32-method | Miscellaneous functions for Column operations
createDataFrame | Create a SparkDataFrame
createExternalTable | (Deprecated) Create an external table
createOrReplaceTempView | Creates a temporary view using the given name.
createOrReplaceTempView-method | Creates a temporary view using the given name.
createTable | Creates a table based on the dataset in a data source
create_array | Non-aggregate functions for Column operations
create_array-method | Non-aggregate functions for Column operations
create_lambda | Create o.a.s.sql.expressions.LambdaFunction corresponding to transformation described by func. Used by higher order functions.
create_map | Non-aggregate functions for Column operations
create_map-method | Non-aggregate functions for Column operations
crossJoin | CrossJoin
crossJoin-method | CrossJoin
crosstab | Computes a pair-wise frequency table of the given columns
crosstab-method | Computes a pair-wise frequency table of the given columns
cube | cube
cube-method | cube
cume_dist | Window functions for Column operations
cume_dist-method | Window functions for Column operations
currentDatabase | Returns the current default database
current_date | Date time functions for Column operations
current_date-method | Date time functions for Column operations
current_timestamp | Date time functions for Column operations
current_timestamp-method | Date time functions for Column operations

## -- D --

dapply | dapply
dapply-method | dapply
dapplyCollect | dapplyCollect
dapplyCollect-method | dapplyCollect
datediff | Date time arithmetic functions for Column operations
datediff-method | Date time arithmetic functions for Column operations
date_add | Date time arithmetic functions for Column operations
date_add-method | Date time arithmetic functions for Column operations
date_format | Date time arithmetic functions for Column operations
date_format-method | Date time arithmetic functions for Column operations
date_sub | Date time arithmetic functions for Column operations
date_sub-method | Date time arithmetic functions for Column operations
date_trunc | Date time functions for Column operations
date_trunc-method | Date time functions for Column operations
dayofmonth | Date time functions for Column operations
dayofmonth-method | Date time functions for Column operations
dayofweek | Date time functions for Column operations
dayofweek-method | Date time functions for Column operations
dayofyear | Date time functions for Column operations
dayofyear-method | Date time functions for Column operations
DecisionTreeClassificationModel-class | S4 class that represents a DecisionTreeClassificationModel
DecisionTreeRegressionModel-class | S4 class that represents a DecisionTreeRegressionModel
decode | String functions for Column operations
decode-method | String functions for Column operations
degrees | Math functions for Column operations
degrees-method | Math functions for Column operations
dense_rank | Window functions for Column operations
dense_rank-method | Window functions for Column operations
desc | A set of operations working with SparkDataFrame columns
describe | describe
describe-method | describe
desc_nulls_first | A set of operations working with SparkDataFrame columns
desc_nulls_last | A set of operations working with SparkDataFrame columns
dim | Returns the dimensions of SparkDataFrame
dim-method | Returns the dimensions of SparkDataFrame
distinct | Distinct
distinct-method | Distinct
drop | drop
drop-method | drop
dropDuplicates | dropDuplicates
dropDuplicates-method | dropDuplicates
dropFields | dropFields
dropFields-method | dropFields
dropna | A set of SparkDataFrame functions working with NA values
dropna-method | A set of SparkDataFrame functions working with NA values
dropTempTable | (Deprecated) Drop Temporary Table
dropTempView | Drops the temporary view with the given view name in the catalog.
dtypes | DataTypes
dtypes-method | DataTypes

## -- E --

element_at | Collection functions for Column operations
element_at-method | Collection functions for Column operations
encode | String functions for Column operations
encode-method | String functions for Column operations
endsWith | endsWith
endsWith-method | endsWith
except | except
except-method | except
exceptAll | exceptAll
exceptAll-method | exceptAll
exp | Math functions for Column operations
exp-method | Math functions for Column operations
explain | Explain
explain-method | Explain
explode | Collection functions for Column operations
explode-method | Collection functions for Column operations
explode_outer | Collection functions for Column operations
explode_outer-method | Collection functions for Column operations
expm1 | Math functions for Column operations
expm1-method | Math functions for Column operations
expr | Non-aggregate functions for Column operations
expr-method | Non-aggregate functions for Column operations

## -- F --

factorial | Math functions for Column operations
factorial-method | Math functions for Column operations
fillna | A set of SparkDataFrame functions working with NA values
fillna-method | A set of SparkDataFrame functions working with NA values
filter | Filter
filter-method | Filter
findFrequentSequentialPatterns-method | PrefixSpan
first | Return the first row of a SparkDataFrame
first-method | Return the first row of a SparkDataFrame
fitted | Get fitted result from a k-means model
fitted-method | Get fitted result from a k-means model
fitted-method | Bisecting K-Means Clustering Model
flatten | Collection functions for Column operations
flatten-method | Collection functions for Column operations
floor | Math functions for Column operations
floor-method | Math functions for Column operations
FMClassificationModel-class | S4 class that represents a FMClassificationModel
FMRegressionModel-class | S4 class that represents a FMRegressionModel
format_number | String functions for Column operations
format_number-method | String functions for Column operations
format_string | String functions for Column operations
format_string-method | String functions for Column operations
FPGrowthModel-class | S4 class that represents a FPGrowthModel
freqItems | Finding frequent items for columns, possibly with false positives
freqItems-method | Finding frequent items for columns, possibly with false positives
freqItemsets-method | FP-growth
from_avro | Avro processing functions for Column operations
from_avro-method | Avro processing functions for Column operations
from_csv | Collection functions for Column operations
from_csv-method | Collection functions for Column operations
from_json | Collection functions for Column operations
from_json-method | Collection functions for Column operations
from_unixtime | Date time functions for Column operations
from_unixtime-method | Date time functions for Column operations
from_utc_timestamp | Date time arithmetic functions for Column operations
from_utc_timestamp-method | Date time arithmetic functions for Column operations

## -- G --

gapply | gapply
gapply-method | gapply
gapplyCollect | gapplyCollect
gapplyCollect-method | gapplyCollect
GaussianMixtureModel-class | S4 class that represents a GaussianMixtureModel
GBTClassificationModel-class | S4 class that represents a GBTClassificationModel
GBTRegressionModel-class | S4 class that represents a GBTRegressionModel
GeneralizedLinearRegressionModel-class | S4 class that represents a generalized linear model
getField | A set of operations working with SparkDataFrame columns
getItem | A set of operations working with SparkDataFrame columns
getLocalProperty | Get a local property set in this thread, or 'NULL' if it is missing. See 'setLocalProperty'.
getNumPartitions | getNumPartitions
getNumPartitions-method | getNumPartitions
glm | Generalized Linear Models (R-compliant)
glm-method | Generalized Linear Models (R-compliant)
greatest | Non-aggregate functions for Column operations
greatest-method | Non-aggregate functions for Column operations
groupBy | GroupBy
groupBy-method | GroupBy
groupedData | S4 class that represents a GroupedData
GroupedData-class | S4 class that represents a GroupedData
grouping_bit | Aggregate functions for Column operations
grouping_bit-method | Aggregate functions for Column operations
grouping_id | Aggregate functions for Column operations
grouping_id-method | Aggregate functions for Column operations
group_by | GroupBy
group_by-method | GroupBy

## -- H --

hash | Miscellaneous functions for Column operations
hash-method | Miscellaneous functions for Column operations
hashCode | Compute the hashCode of an object
head | Head
head-method | Head
hex | Math functions for Column operations
hex-method | Math functions for Column operations
hint | hint
hint-method | hint
histogram | Compute histogram statistics for given column
histogram-method | Compute histogram statistics for given column
hour | Date time functions for Column operations
hour-method | Date time functions for Column operations
hypot | Math functions for Column operations
hypot-method | Math functions for Column operations

## -- I --

ifelse | Non-aggregate functions for Column operations
ifelse-method | Non-aggregate functions for Column operations
initcap | String functions for Column operations
initcap-method | String functions for Column operations
input_file_name | Non-aggregate functions for Column operations
input_file_name-method | Non-aggregate functions for Column operations
insertInto | insertInto
insertInto-method | insertInto
install.spark | Download and Install Apache Spark to a Local Directory
instr | String functions for Column operations
instr-method | String functions for Column operations
intersect | Intersect
intersect-method | Intersect
intersectAll | intersectAll
intersectAll-method | intersectAll
invoke_higher_order_function | Invokes higher order function expression identified by name, (relative to o.a.s.sql.catalyst.expressions)
is.nan | Non-aggregate functions for Column operations
is.nan-method | Non-aggregate functions for Column operations
isActive | isActive
isActive-method | isActive
isLocal | isLocal
isLocal-method | isLocal
isNaN | A set of operations working with SparkDataFrame columns
isnan | Non-aggregate functions for Column operations
isnan-method | Non-aggregate functions for Column operations
isNotNull | A set of operations working with SparkDataFrame columns
isNull | A set of operations working with SparkDataFrame columns
IsotonicRegressionModel-class | S4 class that represents an IsotonicRegressionModel
isStreaming | isStreaming
isStreaming-method | isStreaming

## -- J --

join | Join
join-method | Join

## -- K --

KMeansModel-class | S4 class that represents a KMeansModel
KSTest-class | S4 class that represents an KSTest
kurtosis | Aggregate functions for Column operations
kurtosis-method | Aggregate functions for Column operations

## -- L --

lag | Window functions for Column operations
lag-method | Window functions for Column operations
last | last
last-method | last
lastProgress | lastProgress
lastProgress-method | lastProgress
last_day | Date time functions for Column operations
last_day-method | Date time functions for Column operations
LDAModel-class | S4 class that represents an LDAModel
lead | Window functions for Column operations
lead-method | Window functions for Column operations
least | Non-aggregate functions for Column operations
least-method | Non-aggregate functions for Column operations
length | String functions for Column operations
length-method | String functions for Column operations
levenshtein | String functions for Column operations
levenshtein-method | String functions for Column operations
like | A set of operations working with SparkDataFrame columns
limit | Limit
limit-method | Limit
LinearRegressionModel-class | S4 class that represents a LinearRegressionModel
LinearSVCModel-class | S4 class that represents an LinearSVCModel
listColumns | Returns a list of columns for the given table/view in the specified database
listDatabases | Returns a list of databases available
listFunctions | Returns a list of functions registered in the specified database
listTables | Returns a list of tables or views in the specified database
lit | Non-aggregate functions for Column operations
lit-method | Non-aggregate functions for Column operations
loadDF | Load a SparkDataFrame
localCheckpoint | localCheckpoint
localCheckpoint-method | localCheckpoint
locate | String functions for Column operations
locate-method | String functions for Column operations
log | Math functions for Column operations
log-method | Math functions for Column operations
log10 | Math functions for Column operations
log10-method | Math functions for Column operations
log1p | Math functions for Column operations
log1p-method | Math functions for Column operations
log2 | Math functions for Column operations
log2-method | Math functions for Column operations
LogisticRegressionModel-class | S4 class that represents an LogisticRegressionModel
lower | String functions for Column operations
lower-method | String functions for Column operations
lpad | String functions for Column operations
lpad-method | String functions for Column operations
ltrim | String functions for Column operations
ltrim-method | String functions for Column operations

## -- M --

map_concat | Collection functions for Column operations
map_concat-method | Collection functions for Column operations
map_entries | Collection functions for Column operations
map_entries-method | Collection functions for Column operations
map_filter | Collection functions for Column operations
map_filter-method | Collection functions for Column operations
map_from_arrays | Collection functions for Column operations
map_from_arrays-method | Collection functions for Column operations
map_from_entries | Collection functions for Column operations
map_from_entries-method | Collection functions for Column operations
map_keys | Collection functions for Column operations
map_keys-method | Collection functions for Column operations
map_values | Collection functions for Column operations
map_values-method | Collection functions for Column operations
map_zip_with | Collection functions for Column operations
map_zip_with-method | Collection functions for Column operations
max | Aggregate functions for Column operations
max-method | Aggregate functions for Column operations
md5 | Miscellaneous functions for Column operations
md5-method | Miscellaneous functions for Column operations
mean | Aggregate functions for Column operations
mean-method | Aggregate functions for Column operations
merge | Merges two data frames
merge-method | Merges two data frames
min | Aggregate functions for Column operations
min-method | Aggregate functions for Column operations
minute | Date time functions for Column operations
minute-method | Date time functions for Column operations
monotonically_increasing_id | Non-aggregate functions for Column operations
monotonically_increasing_id-method | Non-aggregate functions for Column operations
month | Date time functions for Column operations
month-method | Date time functions for Column operations
months_between | Date time arithmetic functions for Column operations
months_between-method | Date time arithmetic functions for Column operations
MultilayerPerceptronClassificationModel-class | S4 class that represents a MultilayerPerceptronClassificationModel
mutate | Mutate
mutate-method | Mutate

## -- N --

n | Count
n-method | Count
na.omit | A set of SparkDataFrame functions working with NA values
na.omit-method | A set of SparkDataFrame functions working with NA values
NaiveBayesModel-class | S4 class that represents a NaiveBayesModel
names | Column Names of SparkDataFrame
names-method | Column Names of SparkDataFrame
names<- | Column Names of SparkDataFrame
names<--method | Column Names of SparkDataFrame
nanvl | Non-aggregate functions for Column operations
nanvl-method | Non-aggregate functions for Column operations
ncol | Returns the number of columns in a SparkDataFrame
ncol-method | Returns the number of columns in a SparkDataFrame
negate | Non-aggregate functions for Column operations
negate-method | Non-aggregate functions for Column operations
next_day | Date time arithmetic functions for Column operations
next_day-method | Date time arithmetic functions for Column operations
not | !
not-method | !
nrow | Returns the number of rows in a SparkDataFrame
nrow-method | Returns the number of rows in a SparkDataFrame
nth_value | Window functions for Column operations
nth_value-method | Window functions for Column operations
ntile | Window functions for Column operations
ntile-method | Window functions for Column operations
n_distinct | Aggregate functions for Column operations
n_distinct-method | Aggregate functions for Column operations

## -- O --

orderBy | Ordering Columns in a WindowSpec
orderBy-method | Arrange Rows by Variables
orderBy-method | Ordering Columns in a WindowSpec
otherwise | otherwise
otherwise-method | otherwise
over | over
over-method | over
overlay | String functions for Column operations
overlay-method | String functions for Column operations

## -- P --

partitionBy | partitionBy
partitionBy-method | partitionBy
percentile_approx | Aggregate functions for Column operations
percentile_approx-method | Aggregate functions for Column operations
percent_rank | Window functions for Column operations
percent_rank-method | Window functions for Column operations
persist | Persist
persist-method | Persist
pivot | Pivot a column of the GroupedData and perform the specified aggregation.
pivot-method | Pivot a column of the GroupedData and perform the specified aggregation.
pmod | Math functions for Column operations
pmod-method | Math functions for Column operations
posexplode | Collection functions for Column operations
posexplode-method | Collection functions for Column operations
posexplode_outer | Collection functions for Column operations
posexplode_outer-method | Collection functions for Column operations
PowerIterationClustering-class | S4 class that represents a PowerIterationClustering
predict | Makes predictions from a MLlib model
predict-method | Alternating Least Squares (ALS) for Collaborative Filtering
predict-method | Bisecting K-Means Clustering Model
predict-method | Decision Tree Model for Regression and Classification
predict-method | Factorization Machines Classification Model
predict-method | Factorization Machines Regression Model
predict-method | FP-growth
predict-method | Multivariate Gaussian Mixture Model (GMM)
predict-method | Gradient Boosted Tree Model for Regression and Classification
predict-method | Generalized Linear Models
predict-method | Isotonic Regression Model
predict-method | K-Means Clustering Model
predict-method | Linear Regression Model
predict-method | Logistic Regression Model
predict-method | Multilayer Perceptron Classification Model
predict-method | Naive Bayes Models
predict-method | Random Forest Model for Regression and Classification
predict-method | Accelerated Failure Time (AFT) Survival Regression Model
predict-method | Linear SVM Model
PrefixSpan-class | S4 class that represents a PrefixSpan
print.jobj | Print a JVM object reference.
print.structField | Print a Spark StructField.
print.structType | Print a Spark StructType.
print.summary.DecisionTreeClassificationModel | Decision Tree Model for Regression and Classification
print.summary.DecisionTreeRegressionModel | Decision Tree Model for Regression and Classification
print.summary.GBTClassificationModel | Gradient Boosted Tree Model for Regression and Classification
print.summary.GBTRegressionModel | Gradient Boosted Tree Model for Regression and Classification
print.summary.GeneralizedLinearRegressionModel | Generalized Linear Models
print.summary.KSTest | (One-Sample) Kolmogorov-Smirnov Test
print.summary.RandomForestClassificationModel | Random Forest Model for Regression and Classification
print.summary.RandomForestRegressionModel | Random Forest Model for Regression and Classification
printSchema | Print Schema of a SparkDataFrame
printSchema-method | Print Schema of a SparkDataFrame

## -- Q --

quarter | Date time functions for Column operations
quarter-method | Date time functions for Column operations
queryName | queryName
queryName-method | queryName

## -- R --

radians | Math functions for Column operations
radians-method | Math functions for Column operations
raise_error | Miscellaneous functions for Column operations
raise_error-method | Miscellaneous functions for Column operations
rand | Non-aggregate functions for Column operations
rand-method | Non-aggregate functions for Column operations
randn | Non-aggregate functions for Column operations
randn-method | Non-aggregate functions for Column operations
RandomForestClassificationModel-class | S4 class that represents a RandomForestClassificationModel
RandomForestRegressionModel-class | S4 class that represents a RandomForestRegressionModel
randomSplit | randomSplit
randomSplit-method | randomSplit
rangeBetween | rangeBetween
rangeBetween-method | rangeBetween
rank | Window functions for Column operations
rank-method | Window functions for Column operations
rbind | Union two or more SparkDataFrames
rbind-method | Union two or more SparkDataFrames
read.df | Load a SparkDataFrame
read.jdbc | Create a SparkDataFrame representing the database table accessible via JDBC URL
read.json | Create a SparkDataFrame from a JSON file.
read.ml | Load a fitted MLlib model from the input path.
read.orc | Create a SparkDataFrame from an ORC file.
read.parquet | Create a SparkDataFrame from a Parquet file.
read.stream | Load a streaming SparkDataFrame
read.text | Create a SparkDataFrame from a text file.
recoverPartitions | Recovers all the partitions in the directory of a table and update the catalog
refreshByPath | Invalidates and refreshes all the cached data and metadata for SparkDataFrame containing path
refreshTable | Invalidates and refreshes all the cached data and metadata of the given table
regexp_extract | String functions for Column operations
regexp_extract-method | String functions for Column operations
regexp_replace | String functions for Column operations
regexp_replace-method | String functions for Column operations
registerTempTable | (Deprecated) Register Temporary Table
registerTempTable-method | (Deprecated) Register Temporary Table
rename | rename
rename-method | rename
repartition | Repartition
repartition-method | Repartition
repartitionByRange | Repartition by range
repartitionByRange-method | Repartition by range
repeat_string | String functions for Column operations
repeat_string-method | String functions for Column operations
reverse | Collection functions for Column operations
reverse-method | Collection functions for Column operations
rint | Math functions for Column operations
rint-method | Math functions for Column operations
rlike | A set of operations working with SparkDataFrame columns
rollup | rollup
rollup-method | rollup
round | Math functions for Column operations
round-method | Math functions for Column operations
rowsBetween | rowsBetween
rowsBetween-method | rowsBetween
row_number | Window functions for Column operations
row_number-method | Window functions for Column operations
rpad | String functions for Column operations
rpad-method | String functions for Column operations
rtrim | String functions for Column operations
rtrim-method | String functions for Column operations

## -- S --

sample | Sample
sample-method | Sample
sampleBy | Returns a stratified sample without replacement
sampleBy-method | Returns a stratified sample without replacement
sample_frac | Sample
sample_frac-method | Sample
saveAsTable | Save the contents of the SparkDataFrame to a data source as a table
saveAsTable-method | Save the contents of the SparkDataFrame to a data source as a table
saveDF | Save the contents of SparkDataFrame to a data source.
saveDF-method | Save the contents of SparkDataFrame to a data source.
schema | Get schema object
schema-method | Get schema object
schema_of_csv | Collection functions for Column operations
schema_of_csv-method | Collection functions for Column operations
schema_of_json | Collection functions for Column operations
schema_of_json-method | Collection functions for Column operations
sd | Aggregate functions for Column operations
sd-method | Aggregate functions for Column operations
second | Date time functions for Column operations
second-method | Date time functions for Column operations
select | Select
select-method | Select
selectExpr | SelectExpr
selectExpr-method | SelectExpr
setCheckpointDir | Set checkpoint directory
setCurrentDatabase | Sets the current default database
setJobDescription | Set a human readable description of the current job.
setJobGroup | Assigns a group ID to all the jobs started by this thread until the group ID is set to a different value or cleared.
setLocalProperty | Set a local property that affects jobs submitted from this thread, such as the Spark fair scheduler pool.
setLogLevel | Set new log level
sha1 | Miscellaneous functions for Column operations
sha1-method | Miscellaneous functions for Column operations
sha2 | Miscellaneous functions for Column operations
sha2-method | Miscellaneous functions for Column operations
shiftLeft | Math functions for Column operations
shiftLeft-method | Math functions for Column operations
shiftRight | Math functions for Column operations
shiftRight-method | Math functions for Column operations
shiftRightUnsigned | Math functions for Column operations
shiftRightUnsigned-method | Math functions for Column operations
show | show
show-method | show
showDF | showDF
showDF-method | showDF
shuffle | Collection functions for Column operations
shuffle-method | Collection functions for Column operations
sign | Math functions for Column operations
sign-method | Math functions for Column operations
signum | Math functions for Column operations
signum-method | Math functions for Column operations
sin | Math functions for Column operations
sin-method | Math functions for Column operations
sinh | Math functions for Column operations
sinh-method | Math functions for Column operations
size | Collection functions for Column operations
size-method | Collection functions for Column operations
skewness | Aggregate functions for Column operations
skewness-method | Aggregate functions for Column operations
slice | Collection functions for Column operations
slice-method | Collection functions for Column operations
sort_array | Collection functions for Column operations
sort_array-method | Collection functions for Column operations
soundex | String functions for Column operations
soundex-method | String functions for Column operations
spark.addFile | Add a file or directory to be downloaded with this Spark job on every node.
spark.als | Alternating Least Squares (ALS) for Collaborative Filtering
spark.als-method | Alternating Least Squares (ALS) for Collaborative Filtering
spark.assignClusters | PowerIterationClustering
spark.assignClusters-method | PowerIterationClustering
spark.associationRules | FP-growth
spark.associationRules-method | FP-growth
spark.bisectingKmeans | Bisecting K-Means Clustering Model
spark.bisectingKmeans-method | Bisecting K-Means Clustering Model
spark.decisionTree | Decision Tree Model for Regression and Classification
spark.decisionTree-method | Decision Tree Model for Regression and Classification
spark.findFrequentSequentialPatterns | PrefixSpan
spark.findFrequentSequentialPatterns-method | PrefixSpan
spark.fmClassifier | Factorization Machines Classification Model
spark.fmClassifier-method | Factorization Machines Classification Model
spark.fmRegressor | Factorization Machines Regression Model
spark.fmRegressor-method | Factorization Machines Regression Model
spark.fpGrowth | FP-growth
spark.fpGrowth-method | FP-growth
spark.freqItemsets | FP-growth
spark.freqItemsets-method | FP-growth
spark.gaussianMixture | Multivariate Gaussian Mixture Model (GMM)
spark.gaussianMixture-method | Multivariate Gaussian Mixture Model (GMM)
spark.gbt | Gradient Boosted Tree Model for Regression and Classification
spark.gbt-method | Gradient Boosted Tree Model for Regression and Classification
spark.getSparkFiles | Get the absolute path of a file added through spark.addFile.
spark.getSparkFilesRootDirectory | Get the root directory that contains files added through spark.addFile.
spark.glm | Generalized Linear Models
spark.glm-method | Generalized Linear Models
spark.isoreg | Isotonic Regression Model
spark.isoreg-method | Isotonic Regression Model
spark.kmeans | K-Means Clustering Model
spark.kmeans-method | K-Means Clustering Model
spark.kstest | (One-Sample) Kolmogorov-Smirnov Test
spark.kstest-method | (One-Sample) Kolmogorov-Smirnov Test
spark.lapply | Run a function over a list of elements, distributing the computations with Spark
spark.lda | Latent Dirichlet Allocation
spark.lda-method | Latent Dirichlet Allocation
spark.lm | Linear Regression Model
spark.lm-method | Linear Regression Model
spark.logit | Logistic Regression Model
spark.logit-method | Logistic Regression Model
spark.mlp | Multilayer Perceptron Classification Model
spark.mlp-method | Multilayer Perceptron Classification Model
spark.naiveBayes | Naive Bayes Models
spark.naiveBayes-method | Naive Bayes Models
spark.perplexity | Latent Dirichlet Allocation
spark.perplexity-method | Latent Dirichlet Allocation
spark.posterior | Latent Dirichlet Allocation
spark.posterior-method | Latent Dirichlet Allocation
spark.randomForest | Random Forest Model for Regression and Classification
spark.randomForest-method | Random Forest Model for Regression and Classification
spark.survreg | Accelerated Failure Time (AFT) Survival Regression Model
spark.survreg-method | Accelerated Failure Time (AFT) Survival Regression Model
spark.svmLinear | Linear SVM Model
spark.svmLinear-method | Linear SVM Model
SparkDataFrame-class | S4 class that represents a SparkDataFrame
sparkR.callJMethod | Call Java Methods
sparkR.callJStatic | Call Static Java Methods
sparkR.conf | Get Runtime Config from the current active SparkSession
sparkR.init | (Deprecated) Initialize a new Spark Context
sparkR.newJObject | Create Java Objects
sparkR.session | Get the existing SparkSession or initialize a new SparkSession.
sparkR.session.stop | Stop the Spark Session and Spark Context
sparkR.stop | Stop the Spark Session and Spark Context
sparkR.uiWebUrl | Get the URL of the SparkUI instance for the current active SparkSession
sparkR.version | Get version of Spark on which this application is running
sparkRHive.init | (Deprecated) Initialize a new HiveContext
sparkRSQL.init | (Deprecated) Initialize a new SQLContext
spark_partition_id | Non-aggregate functions for Column operations
spark_partition_id-method | Non-aggregate functions for Column operations
split_string | String functions for Column operations
split_string-method | String functions for Column operations
sql | SQL Query
sqrt | Math functions for Column operations
sqrt-method | Math functions for Column operations
startsWith | startsWith
startsWith-method | startsWith
status | status
status-method | status
stddev | Aggregate functions for Column operations
stddev-method | Aggregate functions for Column operations
stddev_pop | Aggregate functions for Column operations
stddev_pop-method | Aggregate functions for Column operations
stddev_samp | Aggregate functions for Column operations
stddev_samp-method | Aggregate functions for Column operations
stopQuery | stopQuery
stopQuery-method | stopQuery
storageLevel | StorageLevel
storageLevel-method | StorageLevel
str | Compactly display the structure of a dataset
str-method | Compactly display the structure of a dataset
StreamingQuery-class | S4 class that represents a StreamingQuery
struct | Non-aggregate functions for Column operations
struct-method | Non-aggregate functions for Column operations
structField | structField
structField.character | structField
structField.jobj | structField
structType | structType
structType.character | structType
structType.jobj | structType
structType.structField | structType
subset | Subset
subset-method | Subset
substr | substr
substr-method | substr
substring_index | String functions for Column operations
substring_index-method | String functions for Column operations
sum | Aggregate functions for Column operations
sum-method | Aggregate functions for Column operations
sumDistinct | Aggregate functions for Column operations
sumDistinct-method | Aggregate functions for Column operations
summarize | summarize
summarize-method | summarize
summary | summary
summary-method | Alternating Least Squares (ALS) for Collaborative Filtering
summary-method | Bisecting K-Means Clustering Model
summary-method | Decision Tree Model for Regression and Classification
summary-method | Factorization Machines Classification Model
summary-method | Factorization Machines Regression Model
summary-method | Multivariate Gaussian Mixture Model (GMM)
summary-method | Gradient Boosted Tree Model for Regression and Classification
summary-method | Generalized Linear Models
summary-method | Isotonic Regression Model
summary-method | K-Means Clustering Model
summary-method | (One-Sample) Kolmogorov-Smirnov Test
summary-method | Latent Dirichlet Allocation
summary-method | Linear Regression Model
summary-method | Logistic Regression Model
summary-method | Multilayer Perceptron Classification Model
summary-method | Naive Bayes Models
summary-method | Random Forest Model for Regression and Classification
summary-method | Accelerated Failure Time (AFT) Survival Regression Model
summary-method | Linear SVM Model
summary-method | summary

## -- T --

tableNames | Table Names
tables | Tables
tableToDF | Create a SparkDataFrame from a SparkSQL table or view
take | Take the first NUM rows of a SparkDataFrame and return the results as a R data.frame
take-method | Take the first NUM rows of a SparkDataFrame and return the results as a R data.frame
tan | Math functions for Column operations
tan-method | Math functions for Column operations
tanh | Math functions for Column operations
tanh-method | Math functions for Column operations
timestamp_seconds | Date time functions for Column operations
timestamp_seconds-method | Date time functions for Column operations
toDegrees | Math functions for Column operations
toDegrees-method | Math functions for Column operations
toJSON | toJSON
toJSON-method | toJSON
toRadians | Math functions for Column operations
toRadians-method | Math functions for Column operations
to_avro | Avro processing functions for Column operations
to_avro-method | Avro processing functions for Column operations
to_csv | Collection functions for Column operations
to_csv-method | Collection functions for Column operations
to_date | Date time functions for Column operations
to_date-method | Date time functions for Column operations
to_json | Collection functions for Column operations
to_json-method | Collection functions for Column operations
to_timestamp | Date time functions for Column operations
to_timestamp-method | Date time functions for Column operations
to_utc_timestamp | Date time arithmetic functions for Column operations
to_utc_timestamp-method | Date time arithmetic functions for Column operations
transform | Mutate
transform-method | Mutate
transform_keys | Collection functions for Column operations
transform_keys-method | Collection functions for Column operations
transform_values | Collection functions for Column operations
transform_values-method | Collection functions for Column operations
translate | String functions for Column operations
translate-method | String functions for Column operations
trim | String functions for Column operations
trim-method | String functions for Column operations
trunc | Date time functions for Column operations
trunc-method | Date time functions for Column operations

## -- U --

unbase64 | String functions for Column operations
unbase64-method | String functions for Column operations
uncacheTable | Uncache Table
unhex | Math functions for Column operations
unhex-method | Math functions for Column operations
union | Return a new SparkDataFrame containing the union of rows
union-method | Return a new SparkDataFrame containing the union of rows
unionAll | Return a new SparkDataFrame containing the union of rows.
unionAll-method | Return a new SparkDataFrame containing the union of rows.
unionByName | Return a new SparkDataFrame containing the union of rows, matched by column names
unionByName-method | Return a new SparkDataFrame containing the union of rows, matched by column names
unique | Distinct
unique-method | Distinct
unix_timestamp | Date time functions for Column operations
unix_timestamp-method | Date time functions for Column operations
unpersist | Unpersist
unpersist-method | Unpersist
unresolved_named_lambda_var | Create o.a.s.sql.expressions.UnresolvedNamedLambdaVariable, convert it to o.s.sql.Column and wrap with R Column. Used by higher order functions.
upper | String functions for Column operations
upper-method | String functions for Column operations

## -- V --

var | Aggregate functions for Column operations
var-method | Aggregate functions for Column operations
variance | Aggregate functions for Column operations
variance-method | Aggregate functions for Column operations
var_pop | Aggregate functions for Column operations
var_pop-method | Aggregate functions for Column operations
var_samp | Aggregate functions for Column operations
var_samp-method | Aggregate functions for Column operations
vector_to_array | ML functions for Column operations
vector_to_array-method | ML functions for Column operations

## -- W --

weekofyear | Date time functions for Column operations
weekofyear-method | Date time functions for Column operations
when | Non-aggregate functions for Column operations
when-method | Non-aggregate functions for Column operations
where | Filter
where-method | Filter
window | Date time functions for Column operations
window-method | Date time functions for Column operations
windowOrderBy | windowOrderBy
windowOrderBy-method | windowOrderBy
windowPartitionBy | windowPartitionBy
windowPartitionBy-method | windowPartitionBy
WindowSpec-class | S4 class that represents a WindowSpec
with | Evaluate a R expression in an environment constructed from a SparkDataFrame
with-method | Evaluate a R expression in an environment constructed from a SparkDataFrame
withColumn | WithColumn
withColumn-method | WithColumn
withColumnRenamed | rename
withColumnRenamed-method | rename
withField | withField
withField-method | withField
withWatermark | withWatermark
withWatermark-method | withWatermark
write.df | Save the contents of SparkDataFrame to a data source.
write.df-method | Save the contents of SparkDataFrame to a data source.
write.jdbc | Save the content of SparkDataFrame to an external database table via JDBC.
write.jdbc-method | Save the content of SparkDataFrame to an external database table via JDBC.
write.json | Save the contents of SparkDataFrame as a JSON file
write.json-method | Save the contents of SparkDataFrame as a JSON file
write.ml | Saves the MLlib model to the input path
write.ml-method | Alternating Least Squares (ALS) for Collaborative Filtering
write.ml-method | Bisecting K-Means Clustering Model
write.ml-method | Decision Tree Model for Regression and Classification
write.ml-method | Factorization Machines Classification Model
write.ml-method | Factorization Machines Regression Model
write.ml-method | FP-growth
write.ml-method | Multivariate Gaussian Mixture Model (GMM)
write.ml-method | Gradient Boosted Tree Model for Regression and Classification
write.ml-method | Generalized Linear Models
write.ml-method | Isotonic Regression Model
write.ml-method | K-Means Clustering Model
write.ml-method | Latent Dirichlet Allocation
write.ml-method | Linear Regression Model
write.ml-method | Logistic Regression Model
write.ml-method | Multilayer Perceptron Classification Model
write.ml-method | Naive Bayes Models
write.ml-method | Random Forest Model for Regression and Classification
write.ml-method | Accelerated Failure Time (AFT) Survival Regression Model
write.ml-method | Linear SVM Model
write.orc | Save the contents of SparkDataFrame as an ORC file, preserving the schema.
write.orc-method | Save the contents of SparkDataFrame as an ORC file, preserving the schema.
write.parquet | Save the contents of SparkDataFrame as a Parquet file, preserving the schema.
write.parquet-method | Save the contents of SparkDataFrame as a Parquet file, preserving the schema.
write.stream | Write the streaming SparkDataFrame to a data source.
write.stream-method | Write the streaming SparkDataFrame to a data source.
write.text | Save the content of SparkDataFrame in a text file at the specified path.
write.text-method | Save the content of SparkDataFrame in a text file at the specified path.

## -- X --

xxhash64 | Miscellaneous functions for Column operations
xxhash64-method | Miscellaneous functions for Column operations

## -- Y --

year | Date time functions for Column operations
year-method | Date time functions for Column operations

## -- misc --

!-method | !
$ | Select
$-method | Select
$<- | Select
$<--method | Select
%<=>% | %<=>%
%<=>%-method | %<=>%
%in% | Match a column with given values.
%in%-method | Match a column with given values.
[ | Subset
[-method | Subset
[[ | Subset
[[-method | Subset
[[<- | Subset
[[<--method | Subset