Mongo DB Cluster

Mongo DB Cluster#

Adapted from Workshop: geoffswc/MongoDB-Python-Workshop

Section 1: Covid-19 Data Johns Hopkins Overview#

This notebook section will demo how to connect to remote client and work with mongoDB database.

Modules#

%%capture
!pip install pymongo
#!pip install dnspython

import pymongo
from pymongo import MongoClient

Connect to the MongoDB server#

Connect to the published URL for the Johns Hopkins covid-19 dataset hosted on Atlas.

mongodb_url = "mongodb+srv://readonly:readonly@covid-19.hip2i.mongodb.net/covid19"
client = MongoClient(mongodb_url)

List the databases and collections#

Now that we have a connection to the server, we can

list the available databases
select a database
list the available collections within that database
select a collection to query

List the available databases#

# uncomment 
# client.list_database_names()

Select a database#

covid19_db = client.get_database("covid19")

type(covid19_db)

pymongo.database.Database

List database collections#

# uncomment
# covid19_db.list_collection_names()

Select Collection#

countries_summary_cl = covid19_db['countries_summary']

countries_summary_cl

Collection(Database(MongoClient(host=['covid-19-shard-00-00.hip2i.mongodb.net:27017', 'covid-19-shard-00-01.hip2i.mongodb.net:27017', 'covid-19-shard-00-02.hip2i.mongodb.net:27017'], document_class=dict, tz_aware=False, connect=True, authsource='admin', replicaset='covid-19-shard-0', tls=True), 'covid19'), 'countries_summary')

Queries#

Single Record Query#

To inspect just one record from the countries_summary collection, we can use the find_one() command.

# uncomment
# countries_summary_cl.find_one()

Multiple Record Query#

To find multiple records, you can use the find() command along with the limit() method

for r in countries_summary_cl.find().limit(5):
    print(r['country'], r['confirmed'])
    print(r)

KeyboardInterrupt

Counting all documents in a collection#

countries_summary_cl.count_documents({})

Projecting#

The next two cells show examples of choosing which fields to display. By default, all values in the records returned from a query will display. To limit the number of them that are displayed, specify fields names.

Note that once you specify a field to return, only those fields you project will be included in the results. The exception is the “_id” field, which will project by default unless you suppress it.

for r in countries_summary_cl.find({},{'country':1, 'confirmed': 1}).limit(10):
    print(r)

{'_id': ObjectId('62af88c1f59d991d149f722f'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7230'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7231'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7232'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7233'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7234'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7235'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7236'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7237'), 'confirmed': 0, 'country': 'Afghanistan'}
{'_id': ObjectId('62af88c1f59d991d149f7238'), 'confirmed': 0, 'country': 'Afghanistan'}

for r in countries_summary_cl.find({},{'_id': 0, 'country': 1, 'confirmed': 1}).limit(10):
    print(r)

{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}
{'confirmed': 0, 'country': 'Afghanistan'}

Filtering#

The next cells will query based on a

single value
multiple values joined by AND
multiple values joined by OR
query based on date

Single Value Filter#

for r in countries_summary_cl.find({'country': 'Ireland'}, {'country':1, 'confirmed': 1}).limit(5):
    print(r)

{'_id': ObjectId('62af88c3f59d991d14a095bf'), 'confirmed': 0, 'country': 'Ireland'}
{'_id': ObjectId('62af88c3f59d991d14a095c0'), 'confirmed': 0, 'country': 'Ireland'}
{'_id': ObjectId('62af88c3f59d991d14a095c1'), 'confirmed': 0, 'country': 'Ireland'}
{'_id': ObjectId('62af88c3f59d991d14a095c2'), 'confirmed': 0, 'country': 'Ireland'}
{'_id': ObjectId('62af88c3f59d991d14a095c3'), 'confirmed': 0, 'country': 'Ireland'}

Boolean OR query#

for r in countries_summary_cl.find({ '$or' : [ { 'country' : 'Ireland' }, { 'country' : 'India' } ] }).limit(25):
    print(r['country'], r['confirmed'])

India 0
India 0
India 0
India 0
India 0
India 0
India 0
India 0
India 1
India 1
India 1
India 2
India 3
India 3
India 3
India 3
India 3
India 3
India 3
India 3
India 3
India 3
India 3
India 3
India 3

Using IN#

for r in countries_summary_cl.find({'country': { '$in': [ "Ireland", "India" ] } }).limit(5):
    print(r)

{'_id': ObjectId('62af88c3f59d991d14a087df'), 'uids': [356], 'confirmed': 0, 'deaths': 0, 'country': 'India', 'date': datetime.datetime(2020, 1, 22, 0, 0), 'country_iso2s': ['IN'], 'country_iso3s': ['IND'], 'country_codes': [356], 'combined_names': ['India'], 'population': 1380004385, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}
{'_id': ObjectId('62af88c3f59d991d14a087e0'), 'uids': [356], 'confirmed': 0, 'deaths': 0, 'country': 'India', 'date': datetime.datetime(2020, 1, 23, 0, 0), 'country_iso2s': ['IN'], 'country_iso3s': ['IND'], 'country_codes': [356], 'combined_names': ['India'], 'population': 1380004385, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}
{'_id': ObjectId('62af88c3f59d991d14a087e1'), 'uids': [356], 'confirmed': 0, 'deaths': 0, 'country': 'India', 'date': datetime.datetime(2020, 1, 24, 0, 0), 'country_iso2s': ['IN'], 'country_iso3s': ['IND'], 'country_codes': [356], 'combined_names': ['India'], 'population': 1380004385, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}
{'_id': ObjectId('62af88c3f59d991d14a087e2'), 'uids': [356], 'confirmed': 0, 'deaths': 0, 'country': 'India', 'date': datetime.datetime(2020, 1, 25, 0, 0), 'country_iso2s': ['IN'], 'country_iso3s': ['IND'], 'country_codes': [356], 'combined_names': ['India'], 'population': 1380004385, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}
{'_id': ObjectId('62af88c3f59d991d14a087e3'), 'uids': [356], 'confirmed': 0, 'deaths': 0, 'country': 'India', 'date': datetime.datetime(2020, 1, 26, 0, 0), 'country_iso2s': ['IN'], 'country_iso3s': ['IND'], 'country_codes': [356], 'combined_names': ['India'], 'population': 1380004385, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}

Using AND#

# this is analogous to the query in Compass: 
# { "date": new Date('2020-01-22')}
# and
# { '$and' : [{ 'country' : 'Ireland' }, { "date": new Date('2020-01-22')} ] }

import datetime

for r in countries_summary_cl.find({ '$and' : [ 
        { 'country' : 'Ireland' }, 
        { 'date' : datetime.datetime(2020, 1, 23, 0, 0) } ] }):
    print(r)

{'_id': ObjectId('62af88c3f59d991d14a095c0'), 'uids': [372], 'confirmed': 0, 'deaths': 0, 'country': 'Ireland', 'date': datetime.datetime(2020, 1, 23, 0, 0), 'country_iso2s': ['IE'], 'country_iso3s': ['IRL'], 'country_codes': [372], 'combined_names': ['Ireland'], 'population': 4937796, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}

Distinct Values#

countries_summary_cl.distinct("country")[:5]

['Afghanistan', 'Albania', 'Algeria', 'Andorra', 'Angola']

Regular Expressions#

To partially match text, you can use a regular expression. Note that this is a computationally expensive operation and may be too slow to be effective on large text fields in large collections.

For more information: https://docs.mongodb.com/manual/reference/operator/query/regex/

for r in countries_summary_cl.find({'country':{ '$regex': 'land$' } }).limit(2):
    print(r)

{'_id': ObjectId('62af88c2f59d991d14a0490a'), 'uids': [246], 'confirmed': 0, 'deaths': 0, 'country': 'Finland', 'date': datetime.datetime(2020, 1, 22, 0, 0), 'country_iso2s': ['FI'], 'country_iso3s': ['FIN'], 'country_codes': [246], 'combined_names': ['Finland'], 'population': 5540718, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}
{'_id': ObjectId('62af88c2f59d991d14a0490b'), 'uids': [246], 'confirmed': 0, 'deaths': 0, 'country': 'Finland', 'date': datetime.datetime(2020, 1, 23, 0, 0), 'country_iso2s': ['FI'], 'country_iso3s': ['FIN'], 'country_codes': [246], 'combined_names': ['Finland'], 'population': 5540718, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}

You can compile a regular expression with python re module

import re
regx = re.compile("ireland", re.IGNORECASE)
for r in countries_summary_cl.find({'country': { '$regex': regx } }).limit(1):
    print(r)

{'_id': ObjectId('62af88c3f59d991d14a095bf'), 'uids': [372], 'confirmed': 0, 'deaths': 0, 'country': 'Ireland', 'date': datetime.datetime(2020, 1, 22, 0, 0), 'country_iso2s': ['IE'], 'country_iso3s': ['IRL'], 'country_codes': [372], 'combined_names': ['Ireland'], 'population': 4937796, 'recovered': 0, 'confirmed_daily': 0, 'deaths_daily': 0, 'recovered_daily': 0}

Aggregations#

Count: https://docs.mongodb.com/manual/reference/operator/aggregation/count/

Sum: https://docs.mongodb.com/manual/reference/operator/aggregation/sum/

global_and_us_cln = covid19_db['global_and_us']

for agg in global_and_us_cln.aggregate([
    {'$group':{'_id':'$country','count':{'$sum': 1}}},
    {'$limit' : 5 }
]):
    print(agg)

{'_id': 'Burma', 'count': 879}
{'_id': 'Seychelles', 'count': 879}
{'_id': 'Antigua and Barbuda', 'count': 879}
{'_id': 'Marshall Islands', 'count': 879}
{'_id': 'Kiribati', 'count': 879}

Sorting#

To sort in an aggregation pipeline, use the $sort operator. This query will count the number of documents for each country in the collection, sorted in descending order of count, then in ascending order by ID in case of a tie.

https://docs.mongodb.com/manual/reference/operator/aggregation/sort/

for agg in global_and_us_cln.aggregate([
        {'$group':{'_id':'$country','count':{'$sum': 1}}},
        {'$sort' : { 'count' : -1, '_id': 1 } },
        {'$limit' : 5 }
    ]):
    print(agg)

{'_id': 'US', 'count': 2938497}
{'_id': 'China', 'count': 29886}
{'_id': 'Canada', 'count': 14943}
{'_id': 'United Kingdom', 'count': 12306}
{'_id': 'France', 'count': 10548}

Sorting#

To sort results by a field value, you use the $orderby operator. This query will return results sorted first by date, then by country. To reverse the ordering, use -1.

https://docs.mongodb.com/manual/reference/operator/meta/orderby/

Note that this has been replaced by .sort() at the mongo shell

for r in global_and_us_cln.find( {'$query': {}, '$orderby': { 'date' : 1 , 'country': 1} }).limit(5):
    print(r['date'], r['country'])

2020-01-22 00:00:00 Afghanistan
2020-01-22 00:00:00 Albania
2020-01-22 00:00:00 Algeria
2020-01-22 00:00:00 Andorra
2020-01-22 00:00:00 Angola

Aggregations across several records#

for agg in global_and_us_cln.aggregate([
        {'$group':{'_id':'$country','recovered':{'$sum': '$recovered'}}},{'$limit' : 5 }
    ]):
    print(agg)

{'_id': 'Bahrain', 'recovered': 48612087}
{'_id': 'France', 'recovered': 96506536}
{'_id': 'Mozambique', 'recovered': 13140950}
{'_id': 'Tajikistan', 'recovered': 4615407}
{'_id': 'Benin', 'recovered': 1723986}

Pandas dataframes#

You may at some point want to convert your results to pandas dataframes.

Pandas provides a relatively straighforward method to convert mongodb results (as well as other types of JSON or dictionary-based data) into a dataframe. However, keep in mind that you may be cramming a nested, tree-like structure into a tabular data format.

import pandas as pd

df = pd.DataFrame.from_records(global_and_us_cln.find({'country': 'Ireland'}))

The results of this query show how nested data such as dictionaries or lists gets placed into columns. This may or may not be a problem for you, though ther esult it is not a normalized table and may not be amenable to SQL or pandas operations that would work on fields in first normal form (i.e., with single, indivisible values).

df.head()

	_id	uid	country_iso2	country_iso3	country_code	country	combined_name	population	loc	date
0	62af814c2c3fd46394cc19fe	372	IE	IRL	372	Ireland	Ireland	4937796	{'type': 'Point', 'coordinates': [-7.6921, 53....	2020-01-22
1	62af814c2c3fd46394cc19ff	372	IE	IRL	372	Ireland	Ireland	4937796	{'type': 'Point', 'coordinates': [-7.6921, 53....	2020-01-23
2	62af814c2c3fd46394cc1a00	372	IE	IRL	372	Ireland	Ireland	4937796	{'type': 'Point', 'coordinates': [-7.6921, 53....	2020-01-24
3	62af814c2c3fd46394cc1a01	372	IE	IRL	372	Ireland	Ireland	4937796	{'type': 'Point', 'coordinates': [-7.6921, 53....	2020-01-25
4	62af814c2c3fd46394cc1a02	372	IE	IRL	372	Ireland	Ireland	4937796	{'type': 'Point', 'coordinates': [-7.6921, 53....	2020-01-26

For example, the “loc” column contains a dictionary with two keys, ‘type’ and ‘Point’ - where ‘Point’ maps to a list of coordinates

df.iloc[0]['loc']

{'type': 'Point', 'coordinates': [-7.6921, 53.1424]}

Section 2. Upload Data#

You will work with the coronavirus research dataset from the Semantic Scholar team at the Allen Institute:

Go to: https://ai2-semanticscholar-cord-19.s3-us-west-2.amazonaws.com/historical_releases.html
Download a subset of this data: cord-19_2020-03-13.tar.gz (0.3 GB)
Untar it, you should get a folder named: 2020-13-13, within this folder, expand the noncomm-use-subset
Create a new database (see mongodb Client) and upload dataset

import json
import glob

Replace MDB_URL with your mongodb connection string

MDB_URL = 'mongodb+srv://user:password@cluster0.movwmkq.mongodb.net/?retryWrites=true&w=majority'
client = MongoClient(MDB_URL)

Replace path with your own directory path

json_files = []
path = 'noncomm_use_subset/*.json'
for file_path in glob.glob(path):
    with open(file_path) as f:
        json_files.append(json.load(f))        
        

db = client['Covid-19'] # database name .coronavirus

Uncomment below to insert json files. If you ar egetting authentification error, make sure you have the correct user (with write and read) and password

# db['coronavirus'].insert_many(json_files[100:200]) # subset

<pymongo.results.InsertManyResult at 0x13e6f5988>

for r in db['coronavirus'].find().limit(1):
    print(r)

{'_id': ObjectId('62afcea2e538792866b43df8'), 'paper_id': '2271485cae8757f2abdb1c2d012bb892c5421ba4', 'metadata': {'title': '', 'authors': [{'first': 'Peter', 'middle': [], 'last': 'Daszak', 'suffix': '', 'affiliation': {'laboratory': '', 'institution': 'EcoHealth Alliance', 'location': {'addrLine': '460 West 34th Street', 'postCode': '10001', 'settlement': 'New York', 'region': 'NY', 'country': 'USA'}}, 'email': ''}, {'first': 'Kevin', 'middle': ['J'], 'last': 'Olival', 'suffix': '', 'affiliation': {'laboratory': '', 'institution': 'EcoHealth Alliance', 'location': {'addrLine': '460 West 34th Street', 'postCode': '10001', 'settlement': 'New York', 'region': 'NY', 'country': 'USA'}}, 'email': ''}, {'first': 'Hongying', 'middle': [], 'last': 'Li', 'suffix': '', 'affiliation': {'laboratory': '', 'institution': 'EcoHealth Alliance', 'location': {'addrLine': '460 West 34th Street', 'postCode': '10001', 'settlement': 'New York', 'region': 'NY', 'country': 'USA'}}, 'email': ''}]}, 'abstract': [{'text': 'A novel bat-origin coronavirus emerged in Wuhan, China in December 2019 and continues to spread across China and the world. At the time of writing, a massive global response has been implemented to control the disease as it spreads from person to person. Yet the high-risk human-wildlife interactions and interfaces that led to the emergence of SARS-CoV and of 2019-nCoV continue to exist in emerging disease hotspots globally. To prevent the next epidemic and pandemic related to these interfaces, we call for research and investment in three areas: 1) surveillance among wildlife to identify the high-risk pathogens they carry; 2) surveillance among people who have contact with wildlife to identify early spillover events; and 3) improvement of market biosecurity regarding the wildlife trade. As the emergence of a novel virus anywhere can impact the furthest reaches of our connected world, international collaboration among scientists is essential to address these risks and prevent the next pandemic.', 'cite_spans': [], 'ref_spans': [], 'section': 'Abstract'}], 'body_text': [{'text': "The emergence of a novel coronavirus, 2019-nCoV in Wuhan December 2019 has led to a global response to the first epidemic of this decade. It has also highlighted two key issues: First, China's rapid and efficient capacity to identify and investigate a newly emerging disease; and second, our continued global vulnerability to epidemics and pandemics. From the date of the first cluster of cases admitted to a local hospital on December 27, 2019, Chinese scientists identified this disease as a new syndrome, discovered the pathogen as the cause, and reported its genetic sequence to the world in less than 14 days [1, 2] . At the time of writing, we have detailed information on its relationship to other bat coronaviruses, many of which were discovered in a collaboration among EcoHealth Alliance, the Wuhan Institute of Virology, and Duke NUS [3] . We also have data from experimental cell line infections [4] , on the clinical findings [5] , the epidemiology of viral transmission [6] , and on its spread to other countries. These rapid and successful results are a resounding endorsement of China's emergence as a 21st Century scientific superpower, and of a country with a modern and efficient public health system, technically supported by the China CDC with a system of Provincial and City CDCs that are trained in effective outbreak investigation and control [7] .", 'cite_spans': [{'start': 614, 'end': 617, 'text': '[1,', 'ref_id': 'BIBREF0'}, {'start': 618, 'end': 620, 'text': '2]', 'ref_id': 'BIBREF1'}, {'start': 845, 'end': 848, 'text': '[3]', 'ref_id': 'BIBREF2'}, {'start': 908, 'end': 911, 'text': '[4]', 'ref_id': None}, {'start': 939, 'end': 942, 'text': '[5]', 'ref_id': 'BIBREF6'}, {'start': 984, 'end': 987, 'text': '[6]', 'ref_id': None}, {'start': 1367, 'end': 1370, 'text': '[7]', 'ref_id': 'BIBREF9'}], 'ref_spans': [], 'section': 'Biosafety and Health xxx (xxxx) xxx'}, {'text': 'However, despite these efforts, this novel virus was able to spread via infected people traveling within China and internationally. Furthermore, the mechanism of spillover that led to human infection is still uncertain, although it has all the hallmarks of a bat-origin SARS-related coronavirus (SARSr-CoV) [4] . This leads us to two questions that we need to think ⁎ Corresponding author. through even now to get ready to prevent the next 2019-nCoV-like pandemic:', 'cite_spans': [{'start': 307, 'end': 310, 'text': '[4]', 'ref_id': None}], 'ref_spans': [], 'section': 'Biosafety and Health xxx (xxxx) xxx'}, {'text': "The first question is why did this virus spread so rapidly, and how can we deal with this risk? China has changed significantly since the emergence of SARS in 2002-2003. The economy has expanded, and many people have become wealthier, leading to an expansion of air and high-speed train travel. It is estimated that 3 billion individual journeys were planned during the Lunar New Year holiday (Chunyun), around 80 million of these being flights. The flight habits of Chinese citizens have changed alsowith many more flying internationally for business or tourism. In addition, since 2003, there has been an increase in travel between China and African countries, across Southeast Asia and into the New World as part of China's expansion of business and trade. This largely explains why the virus was able to spread out of China much more rapidly than SARS-CoV, which took over 2 months to move to Hong Kong then to the rest of the world.", 'cite_spans': [], 'ref_spans': [], 'section': 'Biosafety and Health xxx (xxxx) xxx'}, {'text': 'The second question is what is the source of this virus? The evidence is fairly clear: Most of the initial cases were linked to a seafood market that also sold butchered livestock meat and some wildlife. The virus itself appears to have a wildlife (bat) origin similar to SARS-CoV, which also emerged in a wildlife market through the interactions of humans and other animals that acted as the intermediate hosts of the virus [3, 8, 9] . With two disease outbreaks originating in China significantly linked to wildlife markets, this is an obvious target for control programs to prevent future epedemics and pandemics. Indeed, there have already been calls from Chinese conservationists, public health leaders and policy makers to reduce the wildlife consumption. However, banning or even reducing the sale of wild game may not be straightforward and it is challenging to change behaviors that are influenced by Chinese culture and traditions. In addition to a strong belief in the purported curative power of wildlife and their by-products, the consumption of the rare and expensive wildlife has become a social statement of wealth boosted by economic growth. Changing these cultural habits will take time, however, recent behavioral questionnaire data suggests a generational transformation with reduced wildlife consumption in the younger generation [10] . There is no doubt, however, that the wildlife trade has an inherent risk of bringing people close to pathogens that wildlife carry, that we have not yet encountered, and that have the potential of leading to the next outbreak.', 'cite_spans': [{'start': 425, 'end': 428, 'text': '[3,', 'ref_id': 'BIBREF2'}, {'start': 429, 'end': 431, 'text': '8,', 'ref_id': 'BIBREF10'}, {'start': 432, 'end': 434, 'text': '9]', 'ref_id': 'BIBREF11'}, {'start': 1351, 'end': 1355, 'text': '[10]', 'ref_id': 'BIBREF12'}], 'ref_spans': [], 'section': 'Biosafety and Health xxx (xxxx) xxx'}, {'text': 'Here, we propose three key steps to helping reduce the risk of a future epidemic similar to that caused by the novel coronavirus 2019-nCoV: 1) Surveillance of wildlife for high-risk pathogens Over the past 10 years collaborating with scientists in China, we collected samples from over 10,000 bats and~2000 other mammals across South China and discovered 52 novel SARSr-CoVs, 122 other β-CoVs, more than 350 novel α-CoVs (including the new Swine Acute Diarrheal Syndrome Coronavirus SADS-CoV), and a new "lineage E" β-CoV clade [11, 12] . We found SARS-related CoVs that can bind to human cells, and that cause SARS-like disease in humanized mouse models that was not prevented with a vaccine candidate against SARS-CoV, and were not treatable with almost all of the monoclonal therapies being developed for SARS [8, [13] [14] [15] [16] . Finally, we showed serological evidence that people living at the wildlife-human interface in rural China were being exposed to SARSrelated coronaviruses, perhaps even the same virus as nCoV-2019, between the emergence of SARS and the current outbreak [17, 18] . Together these data mark wildlife-origin coronaviruses as a \'clear and present danger\'. They also highlight exactly the issue of key concern in the current outbreakthat there is a large diversity of viral strains in wildlife in China with significant potential for emergence in people. Further, we estimate that there are thousands of other CoVs in bats across Southeast Asia, many of which will have pandemic potential. We strongly urge that scientists in these countries work to discover all of these viruses so that we can catalog them, develop a reference library for rapid pathogen identification and risk assessment, and test vaccines and therapies against them [19, 20] .', 'cite_spans': [{'start': 528, 'end': 532, 'text': '[11,', 'ref_id': None}, {'start': 533, 'end': 536, 'text': '12]', 'ref_id': 'BIBREF16'}, {'start': 813, 'end': 816, 'text': '[8,', 'ref_id': 'BIBREF10'}, {'start': 817, 'end': 821, 'text': '[13]', 'ref_id': 'BIBREF17'}, {'start': 822, 'end': 826, 'text': '[14]', 'ref_id': None}, {'start': 827, 'end': 831, 'text': '[15]', 'ref_id': 'BIBREF20'}, {'start': 832, 'end': 836, 'text': '[16]', 'ref_id': 'BIBREF21'}, {'start': 1091, 'end': 1095, 'text': '[17,', 'ref_id': None}, {'start': 1096, 'end': 1099, 'text': '18]', 'ref_id': 'BIBREF24'}, {'start': 1770, 'end': 1774, 'text': '[19,', 'ref_id': 'BIBREF25'}, {'start': 1775, 'end': 1778, 'text': '20]', 'ref_id': 'BIBREF26'}], 'ref_spans': [], 'section': 'Biosafety and Health xxx (xxxx) xxx'}, {'text': "The finding of people in a small sample of rural communities in southern China seropositive for a bat SARSr-CoV suggests that bat-origin coronaviruses commonly spillover in the region [18] . Single cases or small clusters of human infection may evade surveillanceparticularly in regions and countries that border China with less healthcare capacity or rural areas where people don't seek diagnosis or treatment in a timely fashion. Surveillance programs can be designed by local public health authorities to identify communities living in regions with high wildlife diversity and likely high diversity of novel viruses [21] . People with frequent contact with wild or domestic animals related to their livelihood and occupation, and patients presenting acute respiratory infection (ARI) or influenza-like illness (ILI) symptoms with unknown etiology can be included into the surveillance as a cost-effective method to identify novel virus spillovers. This 'pre-outbreak surveillance' strategy can be coordinated with different sectors of public health, healthcare, agriculture and forestry to implement sample collection and testing of wildlife, domestic animals, and people in collaboration with research institutions. These efforts will help identify and characterize viral genetic sequence, identify high-risk human populations with antibodies and cell-mediated immunity responses to wildlife-origin CoVs [22] , as well as the risk factors in human behaviors and living environment through interviews. Evidencebased strategies to reduce risk can then be designed and implemented in the communities where viral spillover is identified.", 'cite_spans': [{'start': 184, 'end': 188, 'text': '[18]', 'ref_id': 'BIBREF24'}, {'start': 619, 'end': 623, 'text': '[21]', 'ref_id': 'BIBREF27'}, {'start': 1408, 'end': 1412, 'text': '[22]', 'ref_id': 'BIBREF28'}], 'ref_spans': [], 'section': '2) Surveillance and risk reduction in people at high risk of contact with wildlife'}, {'text': "The wildlife trade has clearly played a role in the emergence of 2019-nCoV, as well as previous diseases in China (SARS) and across the world (e.g. monkeypox in the USA, Ebola in Africa, salmonellosis in the USA and Europe). China's response in the current outbreak was swift and broad: The index market was closed down immediately and once the virus spread, the wildlife trade was banned temporarily in certain provinces, then nationally. Our recent behavioral risk investigations in China identified low levels of environmental biosecurity and high levels of human-animal contact as key risk factors for zoonotic disease emergence, particularly in local wet and animal markets [23] . While the current wildlife trade bans may help disease control at this moment, to prevent future disease emergence, market biosecurity needs to be improved regarding the hygiene and sanitation facilities and regulations, and the source of animals traded at the market. From a viral emergence perspective, farmed animals are likely a lower risk than wild-caught animals. However, given the size of the wildlife farming industry in China, routine disease surveillance and veterinary care at farms and in transport to markets would need to be improved. The connection between this outbreak and wildlife trade have triggered a strong public opinion against wildlife consumption, and scientists have jointly called for an urgent amendment of the Wildlife Protection Law to standardize and manage wildlife trade as a public health and security issue. It requires collaboration among the State Forestry and Grassland Administration, Ministry of Agriculture and Rural Affairs, State Administration for Market Regulation and public health authorities to tackle this issue as a long-term goal.", 'cite_spans': [{'start': 679, 'end': 683, 'text': '[23]', 'ref_id': 'BIBREF29'}], 'ref_spans': [], 'section': '3) Improve biosecurity of the wildlife trade and animal markets'}, {'text': "Dealing with the risk of future epidemics and pandemics requires a global effort. China is located within a major 'emerging disease hotspot' in Southeast Asia, but there is also a high risk of future diseases originating in sub-Saharan Africa, South Asia and Latin America [24] . Furthermore, the drivers of disease emergence are human activities that are expanding on a global scale, including deforestation, agricultural intensification and the wildlife trade. This has led to an exponential increase in the frequency of animal-human contact and likelihood of novel disease emergence and spread, suggesting that pandemics will become more frequent and more devastating in the future. These threats face all countries because once diseases emerge, they travel rapidly and freely through our global networks of travel and trade. We look forward to continued collaboration among scientists from China and the rest of the world, and to helping develop these new strategies to preventing the next pandemic.", 'cite_spans': [{'start': 273, 'end': 277, 'text': '[24]', 'ref_id': 'BIBREF30'}], 'ref_spans': [], 'section': '3) Improve biosecurity of the wildlife trade and animal markets'}], 'bib_entries': {'BIBREF0': {'ref_id': 'b0', 'title': 'A novel coronavirus from patients with pneumonia in China', 'authors': [{'first': 'N', 'middle': [], 'last': 'Zhu', 'suffix': ''}, {'first': 'D', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'X', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'B', 'middle': [], 'last': 'Yang', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Song', 'suffix': ''}, {'first': 'X', 'middle': [], 'last': 'Zhao', 'suffix': ''}, {'first': 'B', 'middle': [], 'last': 'Huang', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Shi', 'suffix': ''}, {'first': 'R', 'middle': [], 'last': 'Lu', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Niu', 'suffix': ''}, {'first': 'F', 'middle': [], 'last': 'Zhan', 'suffix': ''}, {'first': 'X', 'middle': [], 'last': 'Ma', 'suffix': ''}, {'first': 'D', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Xu', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Wu', 'suffix': ''}, {'first': 'G', 'middle': ['F'], 'last': 'Gao', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Tan', 'suffix': ''}], 'year': 2019, 'venue': '', 'volume': '', 'issn': '', 'pages': '', 'other_ids': {'DOI': ['10.1056/NEJMoa2001017']}}, 'BIBREF1': {'ref_id': 'b1', 'title': 'An outbreak of NCIP (2019-nCoV) infection in China -Wuhan', 'authors': [{'first': 'Q', 'middle': [], 'last': 'Li', 'suffix': ''}], 'year': 2019, 'venue': 'The 2019-nCoV Outbreak Joint Field Epidemiology Investigation Team', 'volume': '2', 'issn': '', 'pages': '79--80', 'other_ids': {}}, 'BIBREF2': {'ref_id': 'b2', 'title': 'Bats are natural reservoirs of SARS-like coronaviruses', 'authors': [{'first': 'W', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'Z', 'middle': [], 'last': 'Shi', 'suffix': ''}, {'first': 'M', 'middle': [], 'last': 'Yu', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Ren', 'suffix': ''}, {'first': 'C', 'middle': [], 'last': 'Smith', 'suffix': ''}, {'first': 'J', 'middle': ['H'], 'last': 'Epstein', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Crameri', 'suffix': ''}, {'first': 'Z', 'middle': [], 'last': 'Hu', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Mceachern', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Field', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Daszak', 'suffix': ''}, {'first': 'B', 'middle': ['T'], 'last': 'Eaton', 'suffix': ''}, {'first': 'S', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'L.-F', 'middle': [], 'last': 'Wang', 'suffix': ''}], 'year': 2005, 'venue': '', 'volume': '310', 'issn': '', 'pages': '676--679', 'other_ids': {'DOI': ['10.1126/science.1118391']}}, 'BIBREF5': {'ref_id': 'b5', 'title': 'Discovery of a novel coronavirus associated with the recent pneumonia outbreak in humans and its potential bat origin, bioRxiv (2020) https', 'authors': [{'first': 'G', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'Z.-L', 'middle': [], 'last': 'Xiao', 'suffix': ''}, {'first': '', 'middle': [], 'last': 'Shi', 'suffix': ''}], 'year': None, 'venue': '', 'volume': '', 'issn': '', 'pages': '', 'other_ids': {'DOI': ['10.1101/2020.01.22.914952']}}, 'BIBREF6': {'ref_id': 'b6', 'title': 'Clinical features of patients infected with 2019 novel coronavirus in', 'authors': [{'first': 'C', 'middle': [], 'last': 'Huang', 'suffix': ''}, {'first': 'Y', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'X', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'L', 'middle': [], 'last': 'Ren', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Zhao', 'suffix': ''}, {'first': 'Y', 'middle': [], 'last': 'Hu', 'suffix': ''}, {'first': 'L', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Fan', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Xu', 'suffix': ''}, {'first': 'X', 'middle': [], 'last': 'Gu', 'suffix': ''}, {'first': 'Z', 'middle': [], 'last': 'Cheng', 'suffix': ''}, {'first': 'T', 'middle': [], 'last': 'Yu', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Xia', 'suffix': ''}, {'first': 'Y', 'middle': [], 'last': 'Wei', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Wu', 'suffix': ''}, {'first': 'X', 'middle': [], 'last': 'Xie', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Yin', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'M', 'middle': [], 'last': 'Liu', 'suffix': ''}, {'first': 'Y', 'middle': [], 'last': 'Xiao', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Gao', 'suffix': ''}, {'first': 'L', 'middle': [], 'last': 'Guo', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Xie', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'R', 'middle': [], 'last': 'Jiang', 'suffix': ''}, {'first': 'Z', 'middle': [], 'last': 'Gao', 'suffix': ''}, {'first': 'Q', 'middle': [], 'last': 'Jin', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'B', 'middle': [], 'last': 'Cao', 'suffix': ''}], 'year': 2020, 'venue': '', 'volume': '', 'issn': '', 'pages': '30183--30188', 'other_ids': {'DOI': ['10.1016/S0140-6736(20)30183-5']}}, 'BIBREF8': {'ref_id': 'b8', 'title': 'A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster', 'authors': [{'first': 'J.-P', 'middle': [], 'last': 'Ip', 'suffix': ''}, {'first': 'V', 'middle': ['C'], 'last': 'Cai', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Cheng', 'suffix': ''}, {'first': 'C', 'middle': ['K'], 'last': 'Chen', 'suffix': ''}, {'first': 'K.-Y', 'middle': [], 'last': 'Hui', 'suffix': ''}, {'first': '', 'middle': [], 'last': 'Yuen', 'suffix': ''}], 'year': 2020, 'venue': 'Lancet', 'volume': '', 'issn': '', 'pages': '30154--30163', 'other_ids': {'DOI': ['10.1016/S0140-6736(20)30154-9']}}, 'BIBREF9': {'ref_id': 'b9', 'title': 'China in action: national strategies to combat against emerging infectious diseases', 'authors': [{'first': 'M', 'middle': [], 'last': 'Han', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Gu', 'suffix': ''}, {'first': 'G', 'middle': ['F'], 'last': 'Gao', 'suffix': ''}, {'first': 'W', 'middle': ['J'], 'last': 'Liu', 'suffix': ''}], 'year': 2017, 'venue': 'Sci. China Life Sci', 'volume': '60', 'issn': '', 'pages': '1383--1385', 'other_ids': {}}, 'BIBREF10': {'ref_id': 'b10', 'title': 'Isolation and characterization of a bat SARSlike coronavirus that uses the ACE2 receptor', 'authors': [{'first': 'X', 'middle': ['Y'], 'last': 'Ge', 'suffix': ''}, {'first': 'J', 'middle': ['L'], 'last': 'Li', 'suffix': ''}, {'first': 'X', 'middle': ['L'], 'last': 'Yang', 'suffix': ''}, {'first': 'A', 'middle': ['A'], 'last': 'Chmura', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Zhu', 'suffix': ''}, {'first': 'J', 'middle': ['H'], 'last': 'Epstein', 'suffix': ''}, {'first': 'J', 'middle': ['K'], 'last': 'Mazet', 'suffix': ''}, {'first': 'B', 'middle': [], 'last': 'Hu', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'C', 'middle': [], 'last': 'Peng', 'suffix': ''}, {'first': 'Y', 'middle': ['J'], 'last': 'Zhang', 'suffix': ''}, {'first': 'C', 'middle': ['M'], 'last': 'Luo', 'suffix': ''}, {'first': 'B', 'middle': [], 'last': 'Tan', 'suffix': ''}, {'first': 'N', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'Y', 'middle': [], 'last': 'Zhu', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Crameri', 'suffix': ''}, {'first': 'S', 'middle': ['Y'], 'last': 'Zhang', 'suffix': ''}, {'first': 'L', 'middle': ['F'], 'last': 'Wang', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Daszak', 'suffix': ''}, {'first': 'Z', 'middle': ['L'], 'last': 'Shi', 'suffix': ''}], 'year': 2013, 'venue': 'Nature', 'volume': '503', 'issn': '', 'pages': '535--538', 'other_ids': {'DOI': ['10.1038/nature12711']}}, 'BIBREF11': {'ref_id': 'b11', 'title': 'Bats, civets and the emergence of SARS, Wildlife and Emerging Zoonotic Diseases: The Biology, Circumstances and Consequences of Cross-Species Transmission', 'authors': [{'first': 'L.-F', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'B', 'middle': ['T'], 'last': 'Eaton', 'suffix': ''}], 'year': 2007, 'venue': '', 'volume': '', 'issn': '', 'pages': '325--344', 'other_ids': {}}, 'BIBREF12': {'ref_id': 'b12', 'title': 'Wildlife consumption and conservation awareness in China: a long way to go', 'authors': [{'first': 'L', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'F', 'middle': [], 'last': 'Yin', 'suffix': ''}], 'year': 2014, 'venue': 'Biodivers. Conserv', 'volume': '23', 'issn': '', 'pages': '2371--2381', 'other_ids': {'DOI': ['10.1007/s10531-014-0708-4']}}, 'BIBREF15': {'ref_id': 'b15', 'title': 'Fatal swine acute diarrhoea syndrome caused by an HKU2-related coronavirus of bat origin', 'authors': [{'first': 'J.-Y.', 'middle': [], 'last': 'Tong', 'suffix': ''}, {'first': '', 'middle': [], 'last': 'Ma', 'suffix': ''}], 'year': 2018, 'venue': 'Nature', 'volume': '556', 'issn': '', 'pages': '255--258', 'other_ids': {'DOI': ['10.1038/s41586-018-0010-9']}}, 'BIBREF16': {'ref_id': 'b16', 'title': 'Discovery of a rich gene pool of bat SARS-related coronaviruses provides new insights into the origin of SARS coronavirus', 'authors': [{'first': 'B', 'middle': [], 'last': 'Hu', 'suffix': ''}, {'first': 'L.-P', 'middle': [], 'last': 'Zeng', 'suffix': ''}, {'first': 'X.-L', 'middle': [], 'last': 'Yang', 'suffix': ''}, {'first': 'X.-Y', 'middle': [], 'last': 'Ge', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'B', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'J.-Z', 'middle': [], 'last': 'Xie', 'suffix': ''}, {'first': 'X.-R', 'middle': [], 'last': 'Shen', 'suffix': ''}, {'first': 'Y.-Z', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'N', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'D.-S', 'middle': [], 'last': 'Luo', 'suffix': ''}, {'first': 'X.-S', 'middle': [], 'last': 'Zheng', 'suffix': ''}, {'first': 'M.-N', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Daszak', 'suffix': ''}, {'first': 'L.-F', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'J', 'middle': [], 'last': 'Cui', 'suffix': ''}, {'first': 'Z.-L', 'middle': [], 'last': 'Shi', 'suffix': ''}], 'year': 2017, 'venue': 'PLoS Pathog', 'volume': '13', 'issn': '', 'pages': '', 'other_ids': {'DOI': ['10.1371/journal.ppat.1006698']}}, 'BIBREF17': {'ref_id': 'b17', 'title': 'Viruses in bats and potential spillover to animals and humans', 'authors': [{'first': 'L', 'middle': ['F'], 'last': 'Wang', 'suffix': ''}, {'first': 'D', 'middle': ['E'], 'last': 'Anderson', 'suffix': ''}], 'year': 2019, 'venue': 'Curr Opin Virol', 'volume': '34', 'issn': '', 'pages': '79--89', 'other_ids': {'DOI': ['10.1016/j.coviro.2018.12.007']}}, 'BIBREF19': {'ref_id': 'b19', 'title': 'Isolation and characterization of a novel bat coronavirus closely related to the direct progenitor of severe acute respiratory syndrome coronavirus', 'authors': [{'first': 'P', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': '', 'middle': [], 'last': 'Daszak', 'suffix': ''}], 'year': 2016, 'venue': 'J. Virol', 'volume': '90', 'issn': '', 'pages': '3253--3256', 'other_ids': {}}, 'BIBREF20': {'ref_id': 'b20', 'title': 'Jumping species-a mechanism for coronavirus persistence and survival', 'authors': [{'first': 'V', 'middle': ['D'], 'last': 'Menachery', 'suffix': ''}, {'first': 'R', 'middle': ['L'], 'last': 'Graham', 'suffix': ''}, {'first': 'R', 'middle': ['S'], 'last': 'Baric', 'suffix': ''}], 'year': 2017, 'venue': 'Curr. Opin. Virol', 'volume': '23', 'issn': '', 'pages': '1--7', 'other_ids': {}}, 'BIBREF21': {'ref_id': 'b21', 'title': 'Cross-neutralization of SARS coronavirus-specific antibodies against bat SARS-like coronaviruses', 'authors': [{'first': 'L', 'middle': ['P'], 'last': 'Zeng', 'suffix': ''}, {'first': 'X', 'middle': ['Y'], 'last': 'Ge', 'suffix': ''}, {'first': 'C', 'middle': [], 'last': 'Peng', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Tai', 'suffix': ''}, {'first': 'S', 'middle': [], 'last': 'Jiang', 'suffix': ''}, {'first': 'L', 'middle': [], 'last': 'Du', 'suffix': ''}, {'first': 'Z', 'middle': ['L'], 'last': 'Shi', 'suffix': ''}], 'year': 2017, 'venue': 'Sci. China Life Sci', 'volume': '60', 'issn': '', 'pages': '1399--1402', 'other_ids': {'DOI': ['10.1007/s11427-017-9189-3']}}, 'BIBREF23': {'ref_id': 'b23', 'title': 'Serological evidence of bat SARS-related coronavirus infection in humans, China', 'authors': [{'first': '', 'middle': [], 'last': 'Shi', 'suffix': ''}], 'year': 2018, 'venue': 'Virol Sin', 'volume': '33', 'issn': '', 'pages': '104--107', 'other_ids': {'DOI': ['10.1007/s12250-018-0012-7']}}, 'BIBREF24': {'ref_id': 'b24', 'title': 'Human-animal interactions and bat coronavirus spillover potential among rural residents in Southern China', 'authors': [{'first': 'H', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'E', 'middle': [], 'last': 'Mendelsohn', 'suffix': ''}, {'first': 'C', 'middle': [], 'last': 'Zong', 'suffix': ''}, {'first': 'W', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'E', 'middle': [], 'last': 'Hagan', 'suffix': ''}, {'first': 'N', 'middle': [], 'last': 'Wang', 'suffix': ''}, {'first': 'S', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Yan', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Huang', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Zhu', 'suffix': ''}], 'year': 2019, 'venue': 'Biosafety and Health', 'volume': '1', 'issn': '', 'pages': '84--90', 'other_ids': {'DOI': ['10.1016/j.bsheal.2019.10.004']}}, 'BIBREF25': {'ref_id': 'b25', 'title': 'The global virome project', 'authors': [{'first': 'D', 'middle': [], 'last': 'Carroll', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Daszak', 'suffix': ''}, {'first': 'N', 'middle': ['D'], 'last': 'Wolfe', 'suffix': ''}, {'first': 'G', 'middle': ['F'], 'last': 'Gao', 'suffix': ''}, {'first': 'C', 'middle': ['M'], 'last': 'Morel', 'suffix': ''}, {'first': 'S', 'middle': [], 'last': 'Morzaria', 'suffix': ''}, {'first': 'A', 'middle': [], 'last': 'Pablos-Méndez', 'suffix': ''}, {'first': 'O', 'middle': [], 'last': 'Tomori', 'suffix': ''}, {'first': 'J', 'middle': ['A'], 'last': 'Mazet', 'suffix': ''}], 'year': 2018, 'venue': 'Science', 'volume': '359', 'issn': '', 'pages': '872--874', 'other_ids': {'DOI': ['10.1016/j.ijid.2016.11.097']}}, 'BIBREF26': {'ref_id': 'b26', 'title': 'A" IV to "Z" IKV: attacks from emerging and re-emerging pathogens', 'authors': [{'first': 'G', 'middle': ['F'], 'last': 'Gao', 'suffix': ''}], 'year': 2018, 'venue': 'Cell', 'volume': '172', 'issn': '', 'pages': '1157--1159', 'other_ids': {'DOI': ['10.1016/j.cell.2018.02.025']}}, 'BIBREF27': {'ref_id': 'b27', 'title': 'Host and viral traits predict zoonotic spillover from mammals', 'authors': [{'first': 'K', 'middle': ['J'], 'last': 'Olival', 'suffix': ''}, {'first': 'P', 'middle': ['R'], 'last': 'Hosseini', 'suffix': ''}, {'first': 'C', 'middle': [], 'last': 'Zambrana-Torrelio', 'suffix': ''}, {'first': 'N', 'middle': [], 'last': 'Ross', 'suffix': ''}, {'first': 'T', 'middle': ['L'], 'last': 'Bogich', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Daszak', 'suffix': ''}], 'year': 2017, 'venue': 'Nature', 'volume': '546', 'issn': '', 'pages': '646--650', 'other_ids': {'DOI': ['10.1038/nature22975']}}, 'BIBREF28': {'ref_id': 'b28', 'title': 'Human T-cell immunity against the emerging and re-emerging viruses', 'authors': [{'first': 'M', 'middle': [], 'last': 'Zhao', 'suffix': ''}, {'first': 'H', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'K', 'middle': [], 'last': 'Liu', 'suffix': ''}, {'first': 'G', 'middle': ['F'], 'last': 'Gao', 'suffix': ''}, {'first': 'W', 'middle': ['J'], 'last': 'Liu', 'suffix': ''}], 'year': 2017, 'venue': 'Sci. China Life Sci', 'volume': '60', 'issn': '', 'pages': '1307--1316', 'other_ids': {'DOI': ['10.1007/s11427-017-9241-3']}}, 'BIBREF29': {'ref_id': 'b29', 'title': 'A qualitative study of zoonotic risk among rural communities in Southern China', 'authors': [{'first': 'H', 'middle': [], 'last': 'Li', 'suffix': ''}, {'first': 'G', 'middle': [], 'last': 'Zhu', 'suffix': ''}, {'first': 'Y', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'L', 'middle': [], 'last': 'Zhang', 'suffix': ''}, {'first': 'E', 'middle': [], 'last': 'Hagan', 'suffix': ''}, {'first': 'M', 'middle': [], 'last': 'Stephanie', 'suffix': ''}, {'first': 'A', 'middle': ['A'], 'last': 'Chmura', 'suffix': ''}, {'first': 'L', 'middle': [], 'last': 'Francisco', 'suffix': ''}, {'first': 'T', 'middle': [], 'last': 'Tai', 'suffix': ''}, {'first': 'M', 'middle': [], 'last': 'Maureen', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Daszak', 'suffix': ''}], 'year': 2020, 'venue': 'International Health', 'volume': '12', 'issn': '', 'pages': '1--9', 'other_ids': {'DOI': ['10.1093/inthealth/ihaa001']}}, 'BIBREF30': {'ref_id': 'b30', 'title': 'Global hotspots and correlates of emerging zoonotic diseases', 'authors': [{'first': 'T', 'middle': [], 'last': 'Allen', 'suffix': ''}, {'first': 'K', 'middle': ['A'], 'last': 'Murray', 'suffix': ''}, {'first': 'C', 'middle': [], 'last': 'Zambrana-Torrelio', 'suffix': ''}, {'first': 'S', 'middle': ['S'], 'last': 'Morse', 'suffix': ''}, {'first': 'C', 'middle': [], 'last': 'Rondinini', 'suffix': ''}, {'first': 'M', 'middle': [], 'last': 'Marco', 'suffix': ''}, {'first': 'N', 'middle': [], 'last': 'Breit', 'suffix': ''}, {'first': 'K', 'middle': ['J'], 'last': 'Olival', 'suffix': ''}, {'first': 'P', 'middle': [], 'last': 'Daszak', 'suffix': ''}], 'year': 2017, 'venue': 'Nat. Commun', 'volume': '8', 'issn': '', 'pages': '', 'other_ids': {'DOI': ['10.1038/s41467-017-00923-8']}}}, 'ref_entries': {'FIGREF0': {'text': 'E-mail address: daszak@ecohealthalliance.org. (Peter Daszak).', 'latex': None, 'type': 'figure'}, 'FIGREF1': {'text': 'http://dx.doi.org/10.1016/j.bsheal.2020.01.003 2590-0536/© 2020 Chinese Medical Association Publishing House. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http:// creativecommons.org/licenses/by-nc-nd/4this article as: P. Daszak, K.J. Olival and H. Li, , Biosafety and Health, http://dx.doi.org/10.1016/j.bsheal.2020.01.003', 'latex': None, 'type': 'figure'}}, 'back_matter': [{'text': 'The authors declare that there are no conflicts of interest.', 'cite_spans': [], 'ref_spans': [], 'section': 'Conflflict of interest statement'}]}

Mongo DB Cluster

Contents

Mongo DB Cluster#

Section 1: Covid-19 Data Johns Hopkins Overview#

Modules#

Connect to the MongoDB server#

List the databases and collections#

List the available databases#

Select a database#

List database collections#

Select Collection#

Queries#

Single Record Query#

Multiple Record Query#

Counting all documents in a collection#

Projecting#

Filtering#

Single Value Filter#

Boolean OR query#

Using IN#

Using AND#

Distinct Values#

Regular Expressions#

Aggregations#

Sorting#

Sorting#

Aggregations across several records#

Pandas dataframes#

Section 2. Upload Data#