scala - Closure not working in Filter method -


i have written following under impression line sort of keyword when iterating & filtering through multiple lines of text. error upon compilation. aware line word in spark code used under other context why code not working. have started learning scala 2 days now. please patient little understanding of syntax & slangs.

can explain why line works in spark code & not in plain scala code? can other line in spark & simple scala code.

plain scala code:

val mystring = """hello world line 1       line 2       line 3""";   println(mystring.filter(line => line.contains(3)));                                    ^^^^^^ error here 

spark code:

val sc = new sparkcontext(conf) val logdata = sc.textfile(logfile, 2).cache() val numas = logdata.filter(line => line.contains("a")).count()

line name of variable binding, it's no way special neither in plain scala nor in spark.

i don't know of spark believe logdata.filter function a function iterates on lines of multi-line string , filters them predicate on line; logdata in case not string spark-provided class.

when instead use scala's filter function provided string iterating on characters, line variable binding of type char, doesn't have contains function.

if want iterate on lines of plain scala string can like:

yourstring.split("\n").filter(line => line.contains('3')) 

the split bit makes array[string] containing lines , can filter array; in case line of type string , type of '3' has char work expected.


Comments

Popular posts from this blog

java - Date formats difference between yyyy-MM-dd'T'HH:mm:ss and yyyy-MM-dd'T'HH:mm:ssXXX -

c# - Get rid of xmlns attribute when adding node to existing xml -