PubMatrixPython — full reference notebook¶

No description has been provided for this image

Complete walkthrough of every parameter and feature. Mirrors the PubMatrixR documentation.

Reference: Becker et al. (2003) BMC Bioinformatics 4:61. https://doi.org/10.1186/1471-2105-4-61

Overview¶

PubMatrixPython performs systematic literature searches on PubMed and PMC databases using pairwise combinations of search terms. It builds co-occurrence matrices showing the number of publications that mention both terms from two different sets, enabling researchers to explore relationships between genes, diseases, pathways, or any other biomedical concepts.

This is a Python port of the PubMatrixR R package, with added concurrency, disk caching, and configurable timeouts.

Key Features¶

  • Pairwise literature search — automatically searches all combinations of terms from two lists
  • Multiple database support — search PubMed or PMC via NCBI E-utilities
  • Heatmap visualisation — overlap-percentage heatmaps with optional hierarchical clustering
  • Export capabilities — save results as CSV (with PubMed hyperlink formulas) or ODS
  • Date filtering — restrict searches to a publication date range
  • Flexible input — pass term lists directly or load them from a text file
  • Concurrency — n_workers for parallel queries, respecting NCBI rate limits
  • Caching — cache_dir persists query results to disk between runs
  • Progress tracking — built-in progress bar for long searches

Use Cases¶

PubMatrixPython is particularly useful for:

  • Gene–disease association studies — explore literature connections between genes and diseases
  • Pathway analysis — investigate co-occurrence of genes within or across biological pathways
  • Drug–target research — analyse relationships between compounds and potential targets
  • Systematic literature reviews — quantify research coverage across multiple topics
  • Knowledge gap identification — find under-researched combinations of terms
  • Bibliometric analysis — measure research activity in specific domains

Setup¶

In [1]:
import sys
sys.path.insert(0, '..')

import pandas as pd
import matplotlib.pyplot as plt

from pubmatrix import (
    pubmatrix,
    pubmatrix_from_file,
    plot_pubmatrix_heatmap,
    pubmatrix_heatmap,
)

NCBI API key¶

Without a key NCBI allows 3 requests/second; with a key, 10/second. Get one at https://account.ncbi.nlm.nih.gov/

API_KEY = "YOUR_KEY_HERE"

Leave as None to run without one.

In [2]:
API_KEY = None  # replace with your key to increase rate limit

pubmatrix() — core query function¶

pubmatrix(
    A,                   # list of str — column terms
    B,                   # list of str — row terms
    api_key   = None,    # NCBI API key
    database  = "pubmed",# "pubmed" or "pmc"
    daterange = None,    # [start_year, end_year]
    outfile   = None,    # base filename for export
    export_format = None,# None | "csv" | "ods"
    n_tries   = 2,       # retries on network failure
    n_workers = 1,       # parallel workers for concurrent queries
    timeout   = 30,      # HTTP request timeout in seconds
    cache_dir = None,    # directory to cache query results on disk
)

Returns a pandas.DataFrame — rows = B terms, columns = A terms, values = publication counts.

Basic usage¶

In [3]:
A = ["WNT1", "WNT2", "CTNNB1"]
B = ["obesity", "diabetes", "cancer"]

result = pubmatrix(A=A, B=B, api_key=API_KEY)
result

Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]
Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.79query/s]
Querying NCBI:  22%|██▏       | 2/9 [00:01<00:03,  1.80query/s]
Querying NCBI:  33%|███▎      | 3/9 [00:01<00:03,  1.79query/s]
Querying NCBI:  44%|████▍     | 4/9 [00:02<00:02,  1.75query/s]
Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.53query/s]
Querying NCBI:  67%|██████▋   | 6/9 [00:03<00:02,  1.41query/s]
Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.51query/s]
Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.44query/s]
Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.51query/s]
Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.55query/s]








Out[3]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451






























Larger matrix — 7 × 7 WNT × obesity genes¶


















In [4]:






wnt_genes     = ["WNT1", "WNT2", "WNT3A", "WNT5A", "WNT7B", "CTNNB1", "DVL1"]
obesity_genes = ["LEPR", "ADIPOQ", "PPARG", "TNF", "IL6", "ADRB2", "INSR"]

result_wnt = pubmatrix(A=wnt_genes, B=obesity_genes, api_key=API_KEY)
result_wnt

























Querying NCBI:   0%|          | 0/49 [00:00<?, ?query/s]












Querying NCBI:   2%|▏         | 1/49 [00:00<00:26,  1.82query/s]












Querying NCBI:   4%|▍         | 2/49 [00:01<00:26,  1.79query/s]












Querying NCBI:   6%|▌         | 3/49 [00:01<00:25,  1.79query/s]












Querying NCBI:   8%|▊         | 4/49 [00:02<00:29,  1.54query/s]












Querying NCBI:  10%|█         | 5/49 [00:03<00:26,  1.63query/s]












Querying NCBI:  12%|█▏        | 6/49 [00:03<00:25,  1.68query/s]












Querying NCBI:  14%|█▍        | 7/49 [00:04<00:26,  1.57query/s]












Querying NCBI:  16%|█▋        | 8/49 [00:04<00:25,  1.62query/s]












Querying NCBI:  18%|█▊        | 9/49 [00:05<00:23,  1.67query/s]












Querying NCBI:  20%|██        | 10/49 [00:05<00:23,  1.69query/s]












Querying NCBI:  22%|██▏       | 11/49 [00:06<00:24,  1.55query/s]












Querying NCBI:  24%|██▍       | 12/49 [00:07<00:22,  1.62query/s]












Querying NCBI:  27%|██▋       | 13/49 [00:08<00:23,  1.51query/s]












Querying NCBI:  29%|██▊       | 14/49 [00:08<00:23,  1.47query/s]












Querying NCBI:  31%|███       | 15/49 [00:09<00:24,  1.39query/s]












Querying NCBI:  33%|███▎      | 16/49 [00:10<00:23,  1.39query/s]












Querying NCBI:  35%|███▍      | 17/49 [00:10<00:21,  1.48query/s]












Querying NCBI:  37%|███▋      | 18/49 [00:11<00:19,  1.56query/s]












Querying NCBI:  39%|███▉      | 19/49 [00:12<00:18,  1.63query/s]












Querying NCBI:  41%|████      | 20/49 [00:12<00:19,  1.51query/s]












Querying NCBI:  43%|████▎     | 21/49 [00:13<00:19,  1.41query/s]












Querying NCBI:  45%|████▍     | 22/49 [00:14<00:17,  1.51query/s]












Querying NCBI:  47%|████▋     | 23/49 [00:14<00:18,  1.43query/s]












Querying NCBI:  49%|████▉     | 24/49 [00:15<00:18,  1.36query/s]












Querying NCBI:  51%|█████     | 25/49 [00:16<00:16,  1.47query/s]












Querying NCBI:  53%|█████▎    | 26/49 [00:16<00:14,  1.56query/s]












Querying NCBI:  55%|█████▌    | 27/49 [00:17<00:14,  1.49query/s]












Querying NCBI:  57%|█████▋    | 28/49 [00:18<00:13,  1.58query/s]












Querying NCBI:  59%|█████▉    | 29/49 [00:18<00:13,  1.47query/s]












Querying NCBI:  61%|██████    | 30/49 [00:19<00:13,  1.39query/s]












Querying NCBI:  63%|██████▎   | 31/49 [00:20<00:12,  1.49query/s]












Querying NCBI:  65%|██████▌   | 32/49 [00:21<00:11,  1.42query/s]












Querying NCBI:  67%|██████▋   | 33/49 [00:21<00:10,  1.51query/s]












Querying NCBI:  69%|██████▉   | 34/49 [00:22<00:10,  1.45query/s]












Querying NCBI:  71%|███████▏  | 35/49 [00:22<00:09,  1.53query/s]












Querying NCBI:  73%|███████▎  | 36/49 [00:23<00:08,  1.60query/s]












Querying NCBI:  76%|███████▌  | 37/49 [00:24<00:07,  1.53query/s]












Querying NCBI:  78%|███████▊  | 38/49 [00:24<00:06,  1.62query/s]












Querying NCBI:  80%|███████▉  | 39/49 [00:25<00:06,  1.66query/s]












Querying NCBI:  82%|████████▏ | 40/49 [00:25<00:05,  1.69query/s]












Querying NCBI:  84%|████████▎ | 41/49 [00:26<00:04,  1.72query/s]












Querying NCBI:  86%|████████▌ | 42/49 [00:27<00:04,  1.58query/s]












Querying NCBI:  88%|████████▊ | 43/49 [00:28<00:04,  1.45query/s]












Querying NCBI:  90%|████████▉ | 44/49 [00:28<00:03,  1.37query/s]












Querying NCBI:  92%|█████████▏| 45/49 [00:29<00:02,  1.43query/s]












Querying NCBI:  94%|█████████▍| 46/49 [00:30<00:01,  1.51query/s]












Querying NCBI:  96%|█████████▌| 47/49 [00:30<00:01,  1.46query/s]












Querying NCBI:  98%|█████████▊| 48/49 [00:31<00:00,  1.53query/s]












Querying NCBI: 100%|██████████| 49/49 [00:31<00:00,  1.60query/s]












Querying NCBI: 100%|██████████| 49/49 [00:31<00:00,  1.53query/s]




















Out[4]:












WNT1
WNT2
WNT3A
WNT5A
WNT7B
CTNNB1
DVL1






LEPR
6
0
0
2
0
4
0




ADIPOQ
2
0
0
6
0
10
0




PPARG
2
3
7
5
1
28
0




TNF
83
4
113
126
7
224
3




IL6
75
7
88
146
10
163
3




ADRB2
1
0
0
2
0
0
0




INSR
1
1
1
1
0
4
0






























database parameter¶
"pubmed" (default) searches MEDLINE abstracts.
"pmc" searches full-text articles in PubMed Central — counts are typically higher.


















In [5]:






result_pmc = pubmatrix(A=A, B=B, database="pmc", api_key=API_KEY)
result_pmc

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.82query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:03,  1.82query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:01<00:03,  1.78query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:02,  1.73query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:02<00:02,  1.74query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:03<00:01,  1.57query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.62query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:04<00:00,  1.66query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.48query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.61query/s]




















Out[5]:












WNT1
WNT2
CTNNB1






obesity
2828
1034
5024




diabetes
4476
1643
7254




cancer
12832
5183
30013


























In [6]:






# Side-by-side comparison
print("PubMed:")
print(result)
print("\nPMC:")
print(result_pmc)

























PubMed:
          WNT1  WNT2  CTNNB1
obesity     65     6      93
diabetes   122    18     272
cancer    1286   301    8451

PMC:
           WNT1  WNT2  CTNNB1
obesity    2828  1034    5024
diabetes   4476  1643    7254
cancer    12832  5183   30013

























daterange parameter¶
Filter results to a publication year range.
Useful for tracking how co-occurrence changes over time.


















In [7]:






result_2000_2010 = pubmatrix(A=A, B=B, daterange=[2000, 2010], api_key=API_KEY)
result_2011_2024 = pubmatrix(A=A, B=B, daterange=[2011, 2024], api_key=API_KEY)

print("2000–2010:")
print(result_2000_2010)
print("\n2011–2024:")
print(result_2011_2024)

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.78query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:03,  1.80query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:01<00:03,  1.78query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.48query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.57query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:03<00:02,  1.47query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.55query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.50query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.43query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.52query/s]
























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:06,  1.23query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:05,  1.23query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:02<00:04,  1.41query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.51query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:03,  1.23query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:04<00:02,  1.23query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:05<00:01,  1.26query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:06<00:00,  1.37query/s]












Querying NCBI: 100%|██████████| 9/9 [00:06<00:00,  1.46query/s]












Querying NCBI: 100%|██████████| 9/9 [00:06<00:00,  1.36query/s]












2000–2010:
          WNT1  WNT2  CTNNB1
obesity      1     0       5
diabetes    11     4      41
cancer     361    82    2100

2011–2024:
          WNT1  WNT2  CTNNB1
obesity     60     6      79
diabetes   103    12     201
cancer     768   169    5349





































Output file naming¶
When outfile and export_format are specified, the result is written to
{outfile}_result.{extension} (.csv or .ods). Each cell contains both the
publication count and a hyperlink to the corresponding PubMed search:


    

  • CSV — Excel-compatible HYPERLINK() formulas
    • 

  • ODS — embedded hyperlinks for LibreOffice/OpenOffice
    • 



Row names come from B, column names from A.






















Export to CSV¶
Saves a .csv where each cell is an Excel HYPERLINK formula linking directly
to the PubMed search for that term pair.


















In [8]:






pubmatrix(A=A, B=B, outfile="output", export_format="csv", api_key=API_KEY)

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.79query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:04,  1.46query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:01<00:03,  1.58query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.62query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.66query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:03<00:01,  1.69query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.54query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.48query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.40query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.51query/s]




















Out[8]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451






























Export to ODS¶
Same as CSV but in OpenDocument Spreadsheet format, with clickable hyperlinks
in LibreOffice / OpenOffice.


odfpy is an optional dependency — install with:


pip install pubmatrixpython[ods]



















In [9]:






pubmatrix(A=A, B=B, outfile="output", export_format="ods", api_key=API_KEY)

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:06,  1.26query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:04,  1.54query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:01<00:03,  1.66query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.51query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.39query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:04<00:02,  1.38query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.34query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.45query/s]












Querying NCBI: 100%|██████████| 9/9 [00:06<00:00,  1.42query/s]












Querying NCBI: 100%|██████████| 9/9 [00:06<00:00,  1.43query/s]




















Out[9]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451






























n_tries — retry on network failure¶
Default is 2. Increase for unstable connections.


















In [10]:






result_retry = pubmatrix(A=A, B=B, n_tries=5, api_key=API_KEY)
result_retry

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.81query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:04,  1.48query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:02<00:04,  1.44query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.55query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.38query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:04<00:02,  1.49query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.54query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.45query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.53query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.50query/s]




















Out[10]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451
































Concurrency, timeout, and caching¶
New in v0.2.0: n_workers for parallel queries, timeout for HTTP requests,
and cache_dir to persist results on disk between runs.






















n_workers — concurrent queries¶
Set n_workers > 1 to fetch multiple term pairs in parallel. NCBI rate limits
(3 req/s without an API key, 10 req/s with one) are respected automatically.


















In [11]:






result_parallel = pubmatrix(A=A, B=B, n_workers=3, n_tries=4, api_key=API_KEY)
result_parallel

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:03<00:24,  3.03s/query]












Querying NCBI: 100%|██████████| 9/9 [00:03<00:00,  3.53query/s]












Querying NCBI: 100%|██████████| 9/9 [00:03<00:00,  2.67query/s]




















Out[11]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451






























timeout — HTTP request timeout¶
Default is 30 seconds. Lower it for fast-fail behaviour, or raise it on slow connections.


















In [12]:






result_timeout = pubmatrix(A=A, B=B, timeout=10, api_key=API_KEY)
result_timeout

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:01<00:09,  1.20s/query]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:05,  1.21query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:02<00:04,  1.41query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.53query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.62query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:04<00:01,  1.52query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.60query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.65query/s]












Querying NCBI: 100%|██████████| 9/9 [00:06<00:00,  1.55query/s]












Querying NCBI: 100%|██████████| 9/9 [00:06<00:00,  1.49query/s]




















Out[12]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451






























cache_dir — disk caching¶
When set, each query result is cached as JSON in cache_dir. Repeated calls
with the same terms/parameters are loaded from disk instead of re-querying NCBI.


















In [13]:






import time

start = time.perf_counter()
result_cached = pubmatrix(A=A, B=B, cache_dir="cache", api_key=API_KEY)
first_run = time.perf_counter() - start

start = time.perf_counter()
result_cached_again = pubmatrix(A=A, B=B, cache_dir="cache", api_key=API_KEY)
second_run = time.perf_counter() - start

print(f"First run (network):  {first_run:.2f}s")
print(f"Second run (cached):  {second_run:.2f}s")
result_cached

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.78query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:02<00:09,  1.34s/query]












Querying NCBI:  33%|███▎      | 3/9 [00:03<00:05,  1.01query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:03<00:04,  1.20query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:04<00:02,  1.36query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:04<00:02,  1.31query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:05<00:01,  1.27query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:06<00:00,  1.26query/s]












Querying NCBI: 100%|██████████| 9/9 [00:07<00:00,  1.38query/s]












Querying NCBI: 100%|██████████| 9/9 [00:07<00:00,  1.25query/s]
























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:02,  2.97query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:00<00:02,  2.95query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:01<00:02,  2.95query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:01<00:01,  2.96query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:01<00:01,  2.95query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:02<00:01,  2.95query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:02<00:00,  2.95query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:02<00:00,  2.95query/s]












Querying NCBI: 100%|██████████| 9/9 [00:03<00:00,  2.96query/s]












Querying NCBI: 100%|██████████| 9/9 [00:03<00:00,  2.95query/s]












First run (network):  7.22s
Second run (cached):  3.05s





















Out[13]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451


























In [14]:






import shutil

shutil.rmtree("cache", ignore_errors=True)



























pubmatrix_from_file() — load terms from a text file¶
File format — A terms first, # separator, then B terms:


WNT1
WNT2
CTNNB1
#
obesity
diabetes
cancer



All keyword arguments are passed through to pubmatrix().


















In [15]:






sample_terms = "WNT1\nWNT2\nCTNNB1\n#\nobesity\ndiabetes\ncancer\n"
with open("sample_terms.txt", "w") as f:
    f.write(sample_terms)

result_file = pubmatrix_from_file("sample_terms.txt", api_key=API_KEY)
result_file

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.81query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:04,  1.54query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:01<00:03,  1.64query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.49query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.58query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:03<00:01,  1.64query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.67query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.57query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.42query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.53query/s]




















Out[15]:












WNT1
WNT2
CTNNB1






obesity
65
6
93




diabetes
122
18
272




cancer
1286
301
8451


























In [16]:






# With optional arguments
result_file_dated = pubmatrix_from_file(
    "sample_terms.txt",
    daterange=[2015, 2024],
    api_key=API_KEY,
)
result_file_dated

























Querying NCBI:   0%|          | 0/9 [00:00<?, ?query/s]












Querying NCBI:  11%|█         | 1/9 [00:00<00:04,  1.66query/s]












Querying NCBI:  22%|██▏       | 2/9 [00:01<00:04,  1.66query/s]












Querying NCBI:  33%|███▎      | 3/9 [00:02<00:04,  1.42query/s]












Querying NCBI:  44%|████▍     | 4/9 [00:02<00:03,  1.53query/s]












Querying NCBI:  56%|█████▌    | 5/9 [00:03<00:02,  1.45query/s]












Querying NCBI:  67%|██████▋   | 6/9 [00:04<00:02,  1.36query/s]












Querying NCBI:  78%|███████▊  | 7/9 [00:04<00:01,  1.44query/s]












Querying NCBI:  89%|████████▉ | 8/9 [00:05<00:00,  1.52query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.57query/s]












Querying NCBI: 100%|██████████| 9/9 [00:05<00:00,  1.51query/s]




















Out[16]:












WNT1
WNT2
CTNNB1






obesity
42
4
68




diabetes
89
11
179




cancer
579
136
4404


























In [17]:






import os
os.remove("sample_terms.txt")



























Heatmap visualisation¶
Cell values show overlap percentage:


overlap = (intersection / union) × 100
union   = row_total + col_total - intersection



This is a Jaccard-style normalisation — it accounts for terms that appear
frequently on their own, so a pair like (CTNNB1, cancer) is not inflated
just because both terms are common.






















pubmatrix_heatmap() — quick plot with defaults¶


















In [18]:






pubmatrix_heatmap(result)





















Out[18]:




(<Figure size 1000x800 with 2 Axes>,
 <Axes: title={'center': 'PubMatrix Results'}>)











No description has been provided for this image























plot_pubmatrix_heatmap() — full control¶
plot_pubmatrix_heatmap(
    matrix,
    title          = "PubMatrix Co-occurrence Heatmap",
    cluster_rows   = True,
    cluster_cols   = True,
    show_numbers   = True,
    color_palette  = None,   # list of hex colours; defaults to red gradient
    filename       = None,   # save to PNG if set
    width          = 10,
    height         = 8,
    scale_font     = True,
    show           = False,  # call plt.show() after plotting
)



Returns (fig, ax) — the figure is not displayed automatically; pass show=True
or call plt.show()/fig yourself in interactive sessions.


















In [19]:






plot_pubmatrix_heatmap(
    result,
    title="WNT Genes × Disease Co-occurrence",
    cluster_rows=True,
    cluster_cols=True,
    show_numbers=True,
    width=8,
    height=5,
)





















Out[19]:




(<Figure size 800x500 with 2 Axes>,
 <Axes: title={'center': 'WNT Genes × Disease Co-occurrence'}>)











No description has been provided for this image























Clustering disabled¶


















In [20]:






plot_pubmatrix_heatmap(
    result,
    title="No clustering",
    cluster_rows=False,
    cluster_cols=False,
)





















Out[20]:




(<Figure size 1000x800 with 2 Axes>, <Axes: title={'center': 'No clustering'}>)











No description has been provided for this image























Numbers hidden¶


















In [21]:






plot_pubmatrix_heatmap(
    result,
    title="No cell annotations",
    show_numbers=False,
)





















Out[21]:




(<Figure size 1000x800 with 2 Axes>,
 <Axes: title={'center': 'No cell annotations'}>)











No description has been provided for this image























Custom colour palette¶
Pass any list of hex colours — gradient is interpolated between them.


















In [22]:






plot_pubmatrix_heatmap(
    result,
    title="Blue gradient",
    color_palette=["#deebf7", "#9ecae1", "#3182bd"],
)





















Out[22]:




(<Figure size 1000x800 with 2 Axes>, <Axes: title={'center': 'Blue gradient'}>)











No description has been provided for this image



















In [23]:






plot_pubmatrix_heatmap(
    result,
    title="Green gradient",
    color_palette=["#e5f5e0", "#a1d99b", "#31a354"],
)





















Out[23]:




(<Figure size 1000x800 with 2 Axes>,
 <Axes: title={'center': 'Green gradient'}>)











No description has been provided for this image























Save to PNG and access (fig, ax)¶
plot_pubmatrix_heatmap() returns (fig, ax) and no longer calls plt.show()
automatically. Use show=True to display interactively, or work with the
returned Figure/Axes directly.


















In [24]:






fig, ax = plot_pubmatrix_heatmap(
    result,
    title="Saved heatmap",
    filename="heatmap_full.png",
    width=8,
    height=5,
    show=True,
)
print(type(fig), type(ax))
























No description has been provided for this image











<class 'matplotlib.figure.Figure'> <class 'matplotlib.axes._axes.Axes'>



























Working with the result DataFrame¶
The return value is a plain pandas.DataFrame — all standard pandas operations apply.






















Summary statistics¶


















In [25]:






print("Column sums (total co-occurrences per A term):")
print(result.sum(axis=0))
print()
print("Row sums (total co-occurrences per B term):")
print(result.sum(axis=1))

























Column sums (total co-occurrences per A term):
WNT1      1473
WNT2       325
CTNNB1    8816
dtype: int64

Row sums (total co-occurrences per B term):
obesity       164
diabetes      412
cancer      10038
dtype: int64

























Bar charts — co-occurrences per term¶


















In [26]:






fig, axes = plt.subplots(1, 2, figsize=(12, 4))

col_totals = result.sum(axis=0).sort_values(ascending=False)
axes[0].bar(col_totals.index, col_totals.values, color="#de2d26")
axes[0].set_title("Co-occurrences per column term (A)")
axes[0].set_ylabel("Total publication count")
axes[0].tick_params(axis="x", rotation=45)

row_totals = result.sum(axis=1).sort_values(ascending=False)
axes[1].bar(row_totals.index, row_totals.values, color="#3182bd")
axes[1].set_title("Co-occurrences per row term (B)")
axes[1].set_ylabel("Total publication count")
axes[1].tick_params(axis="x", rotation=45)

plt.tight_layout()
plt.show()
























No description has been provided for this image























Temporal trend — comparing two date windows¶


















In [27]:






# Reuse results computed above
diff = result_2011_2024 - result_2000_2010
print("Absolute change in co-occurrence counts (2011–2024 vs 2000–2010):")
diff

























Absolute change in co-occurrence counts (2011–2024 vs 2000–2010):









Out[27]:












WNT1
WNT2
CTNNB1






obesity
59
6
74




diabetes
92
8
160




cancer
407
87
3249


























In [28]:






plot_pubmatrix_heatmap(
    diff,
    title="Change in co-occurrences: 2011–2024 vs 2000–2010",
    color_palette=["#f7f7f7", "#fc8d59", "#d73027"],
    cluster_rows=False,
    cluster_cols=False,
    width=7,
    height=4,
)





















Out[28]:




(<Figure size 700x400 with 2 Axes>,
 <Axes: title={'center': 'Change in co-occurrences: 2011–2024 vs 2000–2010'}>)











No description has been provided for this image























Save results to CSV manually¶


















In [29]:






result.to_csv("my_results.csv")
print("Saved.")

























Saved.





















In [ ]:






 



























Performance Notes¶
    

  • Rate limiting: NCBI allows 3 requests/second without an API key, 10/second with one.
    • 

  • Search time: depends on matrix size (A × B combinations), n_workers, and network speed.
    • 

  • Caching: set cache_dir to avoid re-querying NCBI for identical term pairs across runs.
    • 

  • Memory usage: results are held in memory as a pandas.DataFrame; very large matrices
may require substantial RAM.
    • 























Troubleshooting¶
Empty results¶
If many searches return 0 results, try:


    

  • Using broader search terms
    • 

  • Expanding the date range
    • 

  • Checking spelling of scientific terms
    • 

  • Using alternative gene names or synonyms
    • 



Rate limiting errors (HTTP 429)¶
If you encounter RuntimeError from repeated 429 responses:


    

  • Obtain and use an NCBI API key (api_key=...)
    • 

  • Reduce n_workers or the size of your search matrix
    • 

  • Increase n_tries to allow more retries with backoff
    • 



Long search times¶
For large matrices:


    

  • Consider breaking into smaller sub-searches
    • 

  • Use more specific date ranges
    • 

  • Use cache_dir so repeated runs skip already-fetched pairs
    • 

  • Increase n_workers (while respecting rate limits)
    • 























License & Citation¶
This project is licensed under the MIT License — see LICENSE.md for details.


If you use PubMatrixPython in your research, please cite:




Becker KG, Hosack DA, Dennis G Jr, Lempicki RA, Bright TJ, Cheadle C, Engel J.
PubMatrix: a tool for multiplex literature mining.
BMC Bioinformatics. 2003 Dec 10;4:61. https://doi.org/10.1186/1471-2105-4-61




Developers:


    

  • Tyler Laird (Author, original PubMatrixR)
    • 

  • Enrique Toledo (Author, maintainer)
    •