Query bson files using python

I have a bson file: xyz.bson full of useful data and I'd like to query/process the data using python. Is there a simple example/tutorial out there I can get started with?

I don't understand this one.

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

asked Dec 5 '13 at 17:00

dwstein

2,595175488

add a comment |

I have a bson file: xyz.bson full of useful data and I'd like to query/process the data using python. Is there a simple example/tutorial out there I can get started with?

I don't understand this one.

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

asked Dec 5 '13 at 17:00

dwstein

2,595175488

add a comment |

I have a bson file: xyz.bson full of useful data and I'd like to query/process the data using python. Is there a simple example/tutorial out there I can get started with?

I don't understand this one.

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

asked Dec 5 '13 at 17:00

dwstein

2,595175488

I have a bson file: xyz.bson full of useful data and I'd like to query/process the data using python. Is there a simple example/tutorial out there I can get started with?

I don't understand this one.

python mongodb bson

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

asked Dec 5 '13 at 17:00

dwstein

2,595175488

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

asked Dec 5 '13 at 17:00

dwstein

2,595175488

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

edited Dec 5 '13 at 17:11

curtisk

16.3k44764

asked Dec 5 '13 at 17:00

dwstein

2,595175488

asked Dec 5 '13 at 17:00

dwstein

2,595175488

asked Dec 5 '13 at 17:00

dwstein

2,595175488

add a comment |

3 Answers
3

active

oldest

votes

You could use the mongorestore command to import the data into a mongoDB server and then query it by connecting to that server.

edited Dec 5 '13 at 17:30

answered Dec 5 '13 at 17:02

drmirror

3,0372025

the documentation says its for json or csv. does it work with bson? docs.mongodb.org/v2.2/reference/mongoimport

– dwstein
Dec 5 '13 at 17:05

My bad. Yes, it must be mongorestore then. That reads .bson files.

– drmirror
Dec 5 '13 at 17:16

looks like that's the one. i'm a little confused on usage. I have a file on my desktop property.bson and I tried >mongostore --db propertyInfo /Desktop/property.bson and got SytaxError: Unexpected identifier. Can you point me to some good examples?

– dwstein
Dec 5 '13 at 17:58

The command is mongorestore, in your example you wrote mongostore, maybe that is the reason. You may also have to specify both a database name and a collection name to import to, using the -d and -c options (-d is equivalent to --db). You also have an absolute pathname in your example, which may or may not be what you intended.

– drmirror
Dec 5 '13 at 19:18

add a comment |

If you want to stream the data as though it were a flat JSON file on disk rather than loading it into a mongod, you can use this small python-bson-streaming library:

https://github.com/bauman/python-bson-streaming

from bsonstream import KeyValueBSONInput

from sys import argv

for file in argv[1:]:

    f = open(file, 'rb')

    stream = KeyValueBSONInput(fh=f,  fast_string_prematch="somthing") #remove fast string match if not needed

    for id, dict_data in stream:

        if id:

         ...process dict_data...

answered Dec 11 '13 at 21:45

bauman.space

1,200713

That Looks like a very interesting approach for querying The Exportes data

– Dukeatcoding
Feb 17 '14 at 23:15

add a comment |

You may use sonq to query .bson file directly from bash, or you can import and use the lib in Python.

A few examples:

Query a .bson file
sonq -f '{"name": "Stark"}' source.bson

Convert query results to a newline separated .json file
sonq -f '{"name": {"$ne": "Stark"}}' -o target.json source.bson

Query a .bson file in python
from sonq.operation import query_son record_list = list(query_son('source.bson', filters={"name": {"$in": ["Stark"]}}))

answered Nov 16 '18 at 10:24

socrates

51649

add a comment |

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f20406096%2fquery-bson-files-using-python%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

3 Answers
3

active

oldest

votes

3 Answers
3

active

oldest

votes

You could use the mongorestore command to import the data into a mongoDB server and then query it by connecting to that server.

edited Dec 5 '13 at 17:30

answered Dec 5 '13 at 17:02

drmirror

3,0372025

the documentation says its for json or csv. does it work with bson? docs.mongodb.org/v2.2/reference/mongoimport

– dwstein
Dec 5 '13 at 17:05

My bad. Yes, it must be mongorestore then. That reads .bson files.

– drmirror
Dec 5 '13 at 17:16

looks like that's the one. i'm a little confused on usage. I have a file on my desktop property.bson and I tried >mongostore --db propertyInfo /Desktop/property.bson and got SytaxError: Unexpected identifier. Can you point me to some good examples?

– dwstein
Dec 5 '13 at 17:58

The command is mongorestore, in your example you wrote mongostore, maybe that is the reason. You may also have to specify both a database name and a collection name to import to, using the -d and -c options (-d is equivalent to --db). You also have an absolute pathname in your example, which may or may not be what you intended.

– drmirror
Dec 5 '13 at 19:18

add a comment |

You could use the mongorestore command to import the data into a mongoDB server and then query it by connecting to that server.

edited Dec 5 '13 at 17:30

answered Dec 5 '13 at 17:02

drmirror

3,0372025

the documentation says its for json or csv. does it work with bson? docs.mongodb.org/v2.2/reference/mongoimport

– dwstein
Dec 5 '13 at 17:05

My bad. Yes, it must be mongorestore then. That reads .bson files.

– drmirror
Dec 5 '13 at 17:16

looks like that's the one. i'm a little confused on usage. I have a file on my desktop property.bson and I tried >mongostore --db propertyInfo /Desktop/property.bson and got SytaxError: Unexpected identifier. Can you point me to some good examples?

– dwstein
Dec 5 '13 at 17:58

The command is mongorestore, in your example you wrote mongostore, maybe that is the reason. You may also have to specify both a database name and a collection name to import to, using the -d and -c options (-d is equivalent to --db). You also have an absolute pathname in your example, which may or may not be what you intended.

– drmirror
Dec 5 '13 at 19:18

add a comment |

You could use the mongorestore command to import the data into a mongoDB server and then query it by connecting to that server.

edited Dec 5 '13 at 17:30

answered Dec 5 '13 at 17:02

drmirror

3,0372025

You could use the mongorestore command to import the data into a mongoDB server and then query it by connecting to that server.

edited Dec 5 '13 at 17:30

answered Dec 5 '13 at 17:02

drmirror

3,0372025

edited Dec 5 '13 at 17:30

answered Dec 5 '13 at 17:02

drmirror

3,0372025

answered Dec 5 '13 at 17:02

drmirror

3,0372025

answered Dec 5 '13 at 17:02

drmirror

3,0372025

the documentation says its for json or csv. does it work with bson? docs.mongodb.org/v2.2/reference/mongoimport

– dwstein
Dec 5 '13 at 17:05

My bad. Yes, it must be mongorestore then. That reads .bson files.

– drmirror
Dec 5 '13 at 17:16

looks like that's the one. i'm a little confused on usage. I have a file on my desktop property.bson and I tried >mongostore --db propertyInfo /Desktop/property.bson and got SytaxError: Unexpected identifier. Can you point me to some good examples?

– dwstein
Dec 5 '13 at 17:58

The command is mongorestore, in your example you wrote mongostore, maybe that is the reason. You may also have to specify both a database name and a collection name to import to, using the -d and -c options (-d is equivalent to --db). You also have an absolute pathname in your example, which may or may not be what you intended.

– drmirror
Dec 5 '13 at 19:18

add a comment |

the documentation says its for json or csv. does it work with bson? docs.mongodb.org/v2.2/reference/mongoimport

– dwstein
Dec 5 '13 at 17:05

My bad. Yes, it must be mongorestore then. That reads .bson files.

– drmirror
Dec 5 '13 at 17:16

looks like that's the one. i'm a little confused on usage. I have a file on my desktop property.bson and I tried >mongostore --db propertyInfo /Desktop/property.bson and got SytaxError: Unexpected identifier. Can you point me to some good examples?

– dwstein
Dec 5 '13 at 17:58

The command is mongorestore, in your example you wrote mongostore, maybe that is the reason. You may also have to specify both a database name and a collection name to import to, using the -d and -c options (-d is equivalent to --db). You also have an absolute pathname in your example, which may or may not be what you intended.

– drmirror
Dec 5 '13 at 19:18

the documentation says its for json or csv. does it work with bson? docs.mongodb.org/v2.2/reference/mongoimport

– dwstein
Dec 5 '13 at 17:05

My bad. Yes, it must be mongorestore then. That reads .bson files.

– drmirror
Dec 5 '13 at 17:16

looks like that's the one. i'm a little confused on usage. I have a file on my desktop property.bson and I tried >mongostore --db propertyInfo /Desktop/property.bson and got SytaxError: Unexpected identifier. Can you point me to some good examples?

– dwstein
Dec 5 '13 at 17:58

The command is mongorestore, in your example you wrote mongostore, maybe that is the reason. You may also have to specify both a database name and a collection name to import to, using the -d and -c options (-d is equivalent to --db). You also have an absolute pathname in your example, which may or may not be what you intended.

– drmirror
Dec 5 '13 at 19:18

add a comment |

If you want to stream the data as though it were a flat JSON file on disk rather than loading it into a mongod, you can use this small python-bson-streaming library:

https://github.com/bauman/python-bson-streaming

from bsonstream import KeyValueBSONInput

from sys import argv

for file in argv[1:]:

    f = open(file, 'rb')

    stream = KeyValueBSONInput(fh=f,  fast_string_prematch="somthing") #remove fast string match if not needed

    for id, dict_data in stream:

        if id:

         ...process dict_data...

answered Dec 11 '13 at 21:45

bauman.space

1,200713

That Looks like a very interesting approach for querying The Exportes data

– Dukeatcoding
Feb 17 '14 at 23:15

add a comment |

If you want to stream the data as though it were a flat JSON file on disk rather than loading it into a mongod, you can use this small python-bson-streaming library:

https://github.com/bauman/python-bson-streaming

from bsonstream import KeyValueBSONInput

from sys import argv

for file in argv[1:]:

    f = open(file, 'rb')

    stream = KeyValueBSONInput(fh=f,  fast_string_prematch="somthing") #remove fast string match if not needed

    for id, dict_data in stream:

        if id:

         ...process dict_data...

answered Dec 11 '13 at 21:45

bauman.space

1,200713

That Looks like a very interesting approach for querying The Exportes data

– Dukeatcoding
Feb 17 '14 at 23:15

add a comment |

If you want to stream the data as though it were a flat JSON file on disk rather than loading it into a mongod, you can use this small python-bson-streaming library:

https://github.com/bauman/python-bson-streaming

from bsonstream import KeyValueBSONInput

from sys import argv

for file in argv[1:]:

    f = open(file, 'rb')

    stream = KeyValueBSONInput(fh=f,  fast_string_prematch="somthing") #remove fast string match if not needed

    for id, dict_data in stream:

        if id:

         ...process dict_data...

answered Dec 11 '13 at 21:45

bauman.space

1,200713

If you want to stream the data as though it were a flat JSON file on disk rather than loading it into a mongod, you can use this small python-bson-streaming library:

https://github.com/bauman/python-bson-streaming

from bsonstream import KeyValueBSONInput

from sys import argv

for file in argv[1:]:

    f = open(file, 'rb')

    stream = KeyValueBSONInput(fh=f,  fast_string_prematch="somthing") #remove fast string match if not needed

    for id, dict_data in stream:

        if id:

         ...process dict_data...

answered Dec 11 '13 at 21:45

bauman.space

1,200713

answered Dec 11 '13 at 21:45

bauman.space

1,200713

answered Dec 11 '13 at 21:45

bauman.space

1,200713

answered Dec 11 '13 at 21:45

bauman.space

1,200713

That Looks like a very interesting approach for querying The Exportes data

– Dukeatcoding
Feb 17 '14 at 23:15

add a comment |

That Looks like a very interesting approach for querying The Exportes data

– Dukeatcoding
Feb 17 '14 at 23:15

That Looks like a very interesting approach for querying The Exportes data

– Dukeatcoding
Feb 17 '14 at 23:15

add a comment |

You may use sonq to query .bson file directly from bash, or you can import and use the lib in Python.

A few examples:

Query a .bson file
sonq -f '{"name": "Stark"}' source.bson

Convert query results to a newline separated .json file
sonq -f '{"name": {"$ne": "Stark"}}' -o target.json source.bson

Query a .bson file in python
from sonq.operation import query_son record_list = list(query_son('source.bson', filters={"name": {"$in": ["Stark"]}}))

answered Nov 16 '18 at 10:24

socrates

51649

add a comment |

You may use sonq to query .bson file directly from bash, or you can import and use the lib in Python.

A few examples:

Query a .bson file
sonq -f '{"name": "Stark"}' source.bson

Convert query results to a newline separated .json file
sonq -f '{"name": {"$ne": "Stark"}}' -o target.json source.bson

Query a .bson file in python
from sonq.operation import query_son record_list = list(query_son('source.bson', filters={"name": {"$in": ["Stark"]}}))

answered Nov 16 '18 at 10:24

socrates

51649

add a comment |

You may use sonq to query .bson file directly from bash, or you can import and use the lib in Python.

A few examples:

Query a .bson file
sonq -f '{"name": "Stark"}' source.bson

Convert query results to a newline separated .json file
sonq -f '{"name": {"$ne": "Stark"}}' -o target.json source.bson

Query a .bson file in python
from sonq.operation import query_son record_list = list(query_son('source.bson', filters={"name": {"$in": ["Stark"]}}))

answered Nov 16 '18 at 10:24

socrates

51649

You may use sonq to query .bson file directly from bash, or you can import and use the lib in Python.

A few examples:

Query a .bson file
sonq -f '{"name": "Stark"}' source.bson

Convert query results to a newline separated .json file
sonq -f '{"name": {"$ne": "Stark"}}' -o target.json source.bson

Query a .bson file in python
from sonq.operation import query_son record_list = list(query_son('source.bson', filters={"name": {"$in": ["Stark"]}}))

answered Nov 16 '18 at 10:24

socrates

51649

answered Nov 16 '18 at 10:24

socrates

51649

answered Nov 16 '18 at 10:24

socrates

51649

answered Nov 16 '18 at 10:24

socrates

51649

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Vfrdtyky