apache pig - PIG: Do not understand why AVG deos not work when COUNT does -

-

i running following set of commands in pig. data set has 1 row each student in class , each student has number of grades. student name tab separated grades student. scores each student comma separated. need find average grade each student. after grouping, can count of grades each student cannot average score each student. pig complains cannot find iterator when averaging. confused since iterator both aggregate function count , avg same. not sure missing. appreciated?

scripts:

grunt>  = load 'grades.txt' using pigstorage('\t')    (f1:chararray,f2:chararray); grunt> dump a; (s14,59,94,81) (s15,60,77) (s16,77,77) (s17,76,76) (s18,19,61,72) (s20,34,35)  grunt> b = foreach generate f1 stu, flatten(tokenize(f2)) (grade:int); grunt> describe b; b: {stu: chararray,grade: int} grunt> dump b; (s14,59) (s14,94) (s14,81) (s15,60) (s15,77) (s16,77) (s16,77) (s17,76) (s17,76) (s18,19) (s18,61) (s18,72) (s20,34) (s20,35) grunt> grp = group b stu; grunt> cnt = foreach grp generate group, count(b.grade); grunt> dump cnt; (s14,3) (s15,2) (s16,2) (s17,2) (s18,3) (s20,2) grunt> avg = foreach grp generate group, avg(b.grade); grunt> dump avg; 2015-03-20 21:56:30,900 error org.apache.pig.tools.pigstats.pigstatsutil:  1 map  reduce job(s) failed! 2015-03-20 21:56:30,907 error org.apache.pig.tools.grunt.grunt: error 1066:  unable open iterator alias avg details @ logfile: /home/training/pig/pig_1426902869706.log grunt>

as mentioned in comments, workaround found:

changed 

b = foreach generate f1 stu, flatten(tokenize(f2)) (grade:int)

to 

b = foreach generate f1 stu, flatten(tokenize(f2)) grade

and copied bag into:

c = foreach b generate stu stu, grade (int)grade;

Comments

Post a Comment

Popular posts from this blog

java - Could not locate OpenAL library -

-
i writing game using slick2d library, uses lwjgl. lwjgl version 2.9.3. using netbeans 8. i tried adding sounds game, example music track on main menu. problem is, exception when run game, despite game still compiling , running except sound of course: sat mar 21 01:23:57 edt 2015 error:could not locate openal library. org.lwjgl.lwjglexception: not locate openal library. @ org.lwjgl.openal.al.create(al.java:156) @ org.lwjgl.openal.al.create(al.java:156) @ org.lwjgl.openal.al.create(al.java:102) @ org.lwjgl.openal.al.create(al.java:206) @ org.newdawn.slick.openal.soundstore$1.run(soundstore.java:295) @ java.security.accesscontroller.doprivileged(native method) @ org.newdawn.slick.openal.soundstore.init(soundstore.java:292) @ org.newdawn.slick.music.<init>(music.java:156) @ org.newdawn.slick.music.<init>(music.java:75) @ view.mainmenu.init(mainmenu.java:58) i didn't include rest of error states error happens in game code. the...
Read more

c++ - Delete matches in OpenCV (Keypoints and descriptors) -

-
i want check scene image against 2 train images. that, detect features , compute descriptors of both training images. before detecting, computing , matching scene image, delete matches of train1 , train2. because these matches won't facilitate matching of scene image train1 , train2. so, match train1 train2 , vector of matches trainidx , queryidx. how can delete these matches in keypoints-vector , descriptor matrix of train1 , train2? best regards, dwi i have done below: std::vector<cv::keypoint> keypoints[2]; cv::mat descriptor[2]; std::vector< cv::dmatch > matches; /* write code generate keypoints, descriptors , matches here... keypoint[0] -> train image 1 keypoints keypoint[1] -> train image 2 keypoints descriptor[0] -> train image 1 descriptors descriptor[1] -> train image 2 descriptors matches -> matched between train image 1 , 2 */ // logic keep unmatched keypoints , corresponding descriptor...
Read more

sorting - opencl Bitonic sort with 64 bits keys -

-
i used nvidia sdk bitonic sort , works great me. 32 bits (uint) need ulong keys. have typically 2^14 keys @ time , power of 2. searched on not find kernel designed ulong. i tried modified nvidia sdk bitonic sort using ulong keys not work. kernel not crash after de call clenqueuendrangekernel got error : cl_invalid_command_queue can tell me how modify bitonic sort or radixsort, or can sort ulong keys? i running nvidia quadro 4000 opencl 1.1 cuda 6.5.20 full_profile used original nvidia sdk bitonicsort.cl i used"ulong" keys , value input , ouput instead of "uint" thanks helping i made work-around suit need. used 30 bits of key , 18 high bits of value sorting. changed comparators (2 lines): '#define mask 0xffffc000 // suit need from this: if( (*keya > *keyb) == dir ) this: if(((*keya > *keyb)||((*keya == *keyb)&&((*vala & mask)> (*valb & mask)))) == dir ) it works fine. but, still, it's work-a...
Read more