Delete some specific objects from a JSON file using python and then push the file to Elasticsearch Index -


i having large json file contains many object. trying remove number objects fetched file since contains public keys , stuff.. ref have named download.json:

"osfamily": "debian", "sshrsakey":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx2xxx", "lsbmajdistrelease": "32", "interfaces": "eth0,lo", "physicalprocessorcount": 1, "ec2_kernel_id": "pki-724545-724545", 

my python script downloads above json file sqs queue , pushes es server follows :

import os import json import uuid import time import boto.sqs import boto boto.sqs.connection import sqsconnection boto.sqs.message import message boto.sqs.message import rawmessage  sqs = boto.sqs.connect_to_region("ap-southeast-1") q = sqs.get_queue("nishantqueue")  #text_file = open('download.json', 'w') m = q.read(visibility_timeout=15) if m == none:   print "no message!" else:   open('download.json', 'w') json_data:     print m.get_body()     json_data.write(m.get_body())  #    clean_data = json.load(json_data)  ## #    element in clean_data:         ## #      del element['sshdsakey','sshrsakey']  #      json_data.write(clean_data) ##       json_data.close()     q.delete_message(m)     print "push es"     os.system('./push_to_es.sh')     print "cleaning temporary json file"     os.remove('download.json')     print "++++++ successful run +++++++" 

now can see in commented part, trying delete objects don't want , write download.json file , push es . commented part not working way thinking should , throws : ioerror: file not open reading

any appreciated

open file in read write mode.

change:-

with open('download.json', 'w') json_data:

to

with open('download.json', 'r+') json_data:

-------------edit-----------------

why not following:-

clean_data = json.loads(m.get_body()) element in clean_data:   del element['sshdsakey','sshrsakey']   #write clean_data download.json   json_data.write(clean_data)  q.delete_message(m) print "push es" os.system('./push_to_es.sh') print "cleaning temporary json file" os.remove('download.json') print "++++++ successful run +++++++" 

Comments

Popular posts from this blog

java - Date formats difference between yyyy-MM-dd'T'HH:mm:ss and yyyy-MM-dd'T'HH:mm:ssXXX -

c# - Get rid of xmlns attribute when adding node to existing xml -