Executing a cassandra insert query through Python multiprocessing queue -
i have cassandra keyspace sujata
.i connecting cassandra using python driver cassandra.cluster
.the column family of sujata
hello
. following code:-
from multiprocessing import process,queue cassandra.cluster import cluster import os queue=queue() cluster = cluster(['127.0.0.1']) metadata = cluster.metadata session = cluster.connect("sujata") def hi(): global session global queue while true: y=queue.get() if y=="exit": os._exit(0) else: print y session.execute(y) if __name__=="__main__": x=process(target=hi) x.start() in xrange(10): z="insert hello(name) values('" + str(i) + "');" queue.put(z) if i==9: queue.put("exit") session.cluster.shutdown() session.shutdown()
in table, have column name
want insert value of i.the insert query passed through queue.i able contents of queue.when run above code,the output is:-
insert hello(name) values('0');
the session.execute() not working. unable understand why happening.
i don't have cassandra machine, guess, it'll work once move connection part prcoess-function hi()
. like:
def hi(): cluster = cluster(['127.0.0.1']) metadata = cluster.metadata session = cluster.connect("sujata") global queue while true: y=queue.get() if y=="exit": os._exit(0) else: print y session.execute(y)
don't know why it's behaving this, i've seen global
variables behaving oddly new processes.
this not best way guess. because each time it'll connect same database redundant , need closed. hope there might better answer.
edit 1:
i didn't read code properly. using queue
. process hi
start once , queue
being used communicate between processes. connection database once only. don't need database connection in main process. shifting part multiprocess function best way.
Comments
Post a Comment