java - How to print <String, Array[]> as a flat pair? -
setup:
i have data customers , favorite top 10 tv shows. far, able data in javardd<tuple2<string, shows[]>>
. able print , check if expected, is.
objective:
now, need print data file, in following format:
customer_1 fav_tv_show_1 customer_1 fav_tv_show_2 customer_1 fav_tv_show_3 customer_1 fav_tv_show_4 customer_2 fav_tv_show_1 customer_2 fav_tv_show_2 customer_2 fav_tv_show_3 customer_2 fav_tv_show_4 customer_3 fav_tv_show_1 customer_3 fav_tv_show_2 customer_3 fav_tv_show_3 customer_3 fav_tv_show_4
problem:
i don't know how that. far, have tried this:
// need flat pair javapairrdd<string, shows> resultpairs = result.maptopair( new pairfunction<tuple2<string,shows[]>, string, shows>() { public tuple2<string, shows> call(tuple2<string, shows[]> t) { // won't work have return multiple <customer - show> pairs } }); }
any appreciated.
well, it's bit weird got javardd<tuple2<string, shows[]>>
instead of javapairrdd<string, shows[]>
more comfortable work in case of key-value pairs. nonetheless, can follows in order flatten result:
// convert rdd pairrdd format javapairrdd<string, shows[]> pairs = result.maptopair(new pairfunction<tuple2<string,shows[]>, string, shows[]>() { public tuple2<string, shows[]> call(tuple2<string, shows[]> t) throws exception { return t; } }); // flatmap values in order split them respective keys javapairrdd<string, shows> output = pairs.flatmapvalues( new function<shows[], iterable<shows>>() { public iterable<shows> call(shows[] shows) throws exception { return arrays.aslist(shows); } }); // else them output.foreach(new voidfunction<tuple2<string, shows>>() { public void call(tuple2<string, shows> t) throws exception { system.out.println(t._1() + " " + t._2()); } });
alternatively, can obtain output
rdd using flatmaptopair
in 1 step, combining manually array of shows
iterable
follows:
javapairrdd<string, shows> output = result.flatmaptopair( new pairflatmapfunction<tuple2<string, shows[]>, string, shows>() { public iterable<tuple2<string, shows>> call(tuple2<string, shows[]> t) throws exception { arraylist<tuple2<string, shows>> ret = new arraylist<>(); (shows s : t._2()) ret.add(new tuple2<>(t._1(), s)); return ret; } });
hope helped. cheers!
Comments
Post a Comment