java - Running Storm in cluster, cleanup does not get called -
i have following code storm topology, running on cluster:
topologybuilder builder = new topologybuilder(); builder.setspout("word-reader", new wordreader()); builder.setbolt("word-normalizer", new wordnormalizer()) .shufflegrouping("word-reader"); builder.setbolt("word-counter", new wordcounter()).fieldsgrouping( "word-normalizer", new fields("word")); config conf = new config(); conf.put("wordsfile", args[0]); conf.setdebug(false); conf.put(config.topology_max_spout_pending, 1); try { stormsubmitter.submittopology("test-topology", conf, builder.createtopology()); } catch (alreadyaliveexception e) { // todo auto-generated catch block e.printstacktrace(); } catch (invalidtopologyexception e) { // todo auto-generated catch block e.printstacktrace(); } when debug it, following cleanup() method doesn't run in class wordcounter...
@override public void cleanup() { system.out.println("-- word counter --"); } ...and topology doesn't finish running.
cleanup()
is called in local-mode when shutdown() localcluster. not on production cluster when storm kill executed or removed nimbus server (cleanup never called on production cluster, in local mode)
// keep commented if submitting production cluster, since no // automatic shutdown required on production cluster try { // runs specified number of ms after submitting topology on // local cluster , resumes , shutdowns // local cluster thread.sleep(10000); objoflocalcluster.shutdown(); } catch (interruptedexception e) { // prints stack trace throwable object on error // output stream.. e.printstacktrace(); } for more details: go docs
same thing happens
close() (which spout) more details on check here
Comments
Post a Comment