QnaList > Groups > Nutch-User > Sep 2005
faq

MapReduce

Hello,
What do I wrong?
I have setup two systems to setup a MapReduce Nutch Crawler but the jobs 
run always on the same "localhost" when I take a view at http://server:7845
seeds/urls is an url list with 500.000 urls
# put seed directory in ndfs
bin/nutch ndfs -put seeds seeds
# crawl a bit
bin/nutch crawl seeds -depth 10
I had mapred.map.tasks on 2 and on 4 and 8
I had mapred.reduce.tasks on 1 2 and 3
But always I have the same rsult
Name	Host	# running tasks	Secs since heartbeat
tracker_29414	srv34	0	1
tracker_36968	srv21	1	2
srv21 is the master
Thanks,
Paul

asked Sep 29 2005 at 18:22

Paul van Brouwershaven's gravatar image



8 Replies for : MapReduce
Paul van Brouwershaven wrote:
> The AcceptEnv option is only avalible with ssh 3.9 > Debian currently 
> only has 3.8.1p1 in stable and testing. (4.2 unstable)
> 
> Is there an other way to solve the env. problem?
I don't know.  The Fedora and Debian systems that I use have AcceptEnv.
Doug

answered Oct 3 2005 at 20:58

Doug Cutting's gravatar image


check out sample at http://www.mongodb.org/display/DOCS/MapReduce#MapReduce-ShellExample1 
On Jan 5, 10:13 pm, Slevoaca  wrote: 
> Hello 
> 
> I found this very interesting tutorial about mongodb mapreduce:http://blog.mongovue.com/2010/11/03/yet-another-mongodb-map-reduce-tu... 
- show quoted text -

answered Jan 5 2011 at 08:43

Nat's gravatar image


Yes, but i need to know how the finalize function has to be executed in the mapreduce function.
2011/1/5 Nat 
- show quoted text -
--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to [email protected].
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

answered Jan 5 2011 at 10:32

Slevoaca Florin's gravatar image


f = function(){ finalise here }
db.events.mapReduce(m, r, { query : {type:'sale'} }, f)
- show quoted text -
-- 
Bow Chicka Bow Wow

answered Jan 5 2011 at 13:43

Sam Millman's gravatar image


Oops sorry didn't look at the top of the document. It should be this:
f = function(){ finalise here }
db.events.mapReduce(m, r {query:{type:'sale'}, finalize:f})
- show quoted text -

answered Jan 5 2011 at 13:45

Sam Millman's gravatar image


Thanks. Now, in mongoVUE, do you know how to use the "Map Reduce " panel ? I am writing the map, reduce and finalize functions in the specific fields, but when I hit "Go" it starts executing and never stops . Troubleing....
2011/1/5 Sam Millman 
- show quoted text -

answered Jan 6 2011 at 03:22

Slevoaca Florin's gravatar image


Unfortunately not you will need to ask the one who made mongoVUE. I use terminal...
- show quoted text -

answered Jan 6 2011 at 04:42

Sam Millman's gravatar image


Hi, 
Send me your functions and collection information, I'll take a look. 
Regards, 
Ishaan 
Developer, MongoVUE 
On Jan 6, 3:15 pm, Slevoaca Florin  wrote: 
> Thanks. Now, in mongoVUE, do you know how to use the "Map Reduce " panel ? I 
> am writing the map, reduce and finalize functions in the specific fields, 
> but when I hit "Go" it starts executing and never stops . Troubleing.... 
> 
> 2011/1/5 Sam Millman  
> 
> 
> 
> 
> 
> 
> 
> > Oops sorry didn't look at the top of the document. It should be this: 
> 
> > f = function(){ finalise here } 
> 
> > db.events.mapReduce(m, r {query:{type:'sale'}, finalize:f}) 
> 
> > On 5 January 2011 20:36, Sam Millman  wrote: 
> 
> >> f = function(){ finalise here } 
> 
> >> db.events.mapReduce(m, r, { query : {type:'sale'} }, f) 
> 
> >> On 5 January 2011 17:25, Slevoaca Florin wrote: 
> 
> >>> Yes, but i need to know how the finalize function has to be executed in 
> >>> the mapreduce function. 
> 
> >>> 2011/1/5 Nat  
> 
> >>> check out sample at 
> >>>>http://www.mongodb.org/display/DOCS/MapReduce#MapReduce-ShellExample1 
> 
> >>>> On Jan 5, 10:13 pm, Slevoaca  wrote: 
> >>>> > Hello 
> 
> >>>> > I found this very interesting tutorial about mongodb mapreduce: 
> >>>>http://blog.mongovue.com/2010/11/03/yet-another-mongodb-map-reduce-tu. 
> >>>> .. 
> >>>> > . 
> 
> >>>> > There are 3 functions: MapCode, ReduceCode and Finalize. I wrote them 
> >>>> > line by line in mongodb shell, but I don't know how to execute them. 
> >>>> > Could You please tell me how should I do exactly ? Is it something 
> >>>> > like this ? : 
> 
> >>>> >  res = db.cities.mapReduce(MapCode,ReduceCode, {finalize:finalize}); 
> >>>> > db[res.result].find(); 
> 
> >>>> > Or could You please explain me how should I execute them in mongoVUE? 
> >>>> > I see there is a "Map Reduce" field and I wrote the functions there 
> >>>> > but when I hit "Go" it doesn't work. 
> 
> >>>> > Please help me ! 
> 
> >>>> -- 
> >>>> You received this message because you are subscribed to the Google 
> >>>> Groups "mongodb-user" group. 
> >>>> To post to this group, send email to [email protected]. 
> >>>> To unsubscribe from this group, send email to 
> >>>> [email protected] 
> >>>> . 
> >>>> For more options, visit this group at 
> >>>>http://groups.google.com/group/mongodb-user?hl=en. 
> 
> >>>  -- 
> >>> You received this message because you are subscribed to the Google Groups 
> >>> "mongodb-user" group. 
> >>> To post to this group, send email to [email protected]. 
> >>> To unsubscribe from this group, send email to 
> >>> [email protected] 
> >>> . 
> >>> For more options, visit this group at 
> >>>http://groups.google.com/group/mongodb-user?hl=en. 
> 
> >> -- 
> >> Bow Chicka Bow Wow 
> 
> > -- 
> > Bow Chicka Bow Wow 
> 
> > -- 
> > You received this message because you are subscribed to the Google Groups 
> > "mongodb-user" group. 
> > To post to this group, send email to [email protected]. 
> > To unsubscribe from this group, send email to 
> > [email protected] 
> > . 
- show quoted text -

answered Jan 10 2011 at 00:55

Ishaan's gravatar image


Related discussions

Tagged

Group Nutch-user

asked Sep 29 2005 at 18:22

active Jan 10 2011 at 00:55

posts:9

users:6

©2013 QnaList.com . QnaList is part of ZisaTechnologies LLC.