stream - Incremental processing and enrichment with storm -


we have system designed processing social media content. in our storm topology have bolts process, such sentiment analysis, language detection, spam detection , on. tutorials , examples prepared on storm, have seen bolt can emit tuple fields has declared in declareoutputfields() methods. there option emit current bolt's field input tuple?

for example have input tuple contains fields below:

<

text : bla bla

username: paul

date: 01.01.2013

source:twitter

>

i want define output tuple as:

<

text : bla bla

username: paul

date: 01.01.2013

source:twitter

lang:tr

>

note want bolts don't need know before bolt's output tuple schema.

thank you.

you achieve writing function returns bolt given input rather writing bolt directly. parameterize creation of bolt writing function return bolt object output fields want.

obviously have done @ time deploy topology, can't dynamic on stream @ run time, can dynamic @ startup time. like

(defn make-bolt [bolt-name input-fields]   (defbolt bolt-name input-fields     ...))  ....  (topology  {} ;; spouts  {"a-bolt" (bolt-spec {"a-spout":shuffle}                       (make-bolt bolt-name ["input" "tuple" "lang"])))) 

Comments

Popular posts from this blog

linux - Does gcc have any options to add version info in ELF binary file? -

android - send complex objects as post php java -

charts - What graph/dashboard product is facebook using in Dashboard: PUE & WUE -