scala - Closure not working in Filter method -
i have written following under impression line
sort of keyword when iterating & filtering through multiple lines of text. error upon compilation. aware line
word in spark
code used under other context why code not working. have started learning scala 2 days now. please patient little understanding of syntax & slangs.
can explain why line
works in spark code & not in plain scala code? can other line
in spark & simple scala code.
plain scala code:
val mystring = """hello world line 1 line 2 line 3"""; println(mystring.filter(line => line.contains(3))); ^^^^^^ error here
spark code:
val sc = new sparkcontext(conf) val logdata = sc.textfile(logfile, 2).cache() val numas = logdata.filter(line => line.contains("a")).count()
line
name of variable binding, it's no way special neither in plain scala nor in spark.
i don't know of spark believe logdata.filter
function a function iterates on lines of multi-line string , filters them predicate on line; logdata
in case not string spark-provided class.
when instead use scala's filter
function provided string
iterating on characters, line
variable binding of type char
, doesn't have contains
function.
if want iterate on lines of plain scala string can like:
yourstring.split("\n").filter(line => line.contains('3'))
the split
bit makes array[string]
containing lines , can filter array; in case line of type string
, type of '3'
has char
work expected.
Comments
Post a Comment