This project examines the Annals of Eugenics, a scientific journal published from 1925 to 1954 before being renamed the Annals of Human Genetics. Through quantitative analysis of publication metadata, this study explores patterns in authorship, research topics, and publication trends during a controversial period in the history of genetics and statistics.
This analysis is part of a larger investigation into the history of eugenics in academia, particularly at The College of Idaho and within the state of Idaho. Understanding what was published in the Annals of Eugenics provides insight into the type of research that was considered legitimate science during this era and how the field evolved over time.
Data Source: The Annals of Eugenics online archive
Collection Method: Web scraping using Python (BeautifulSoup, requests)
Time Period: 1925-1954 (Volumes 1-18)
Data Points: Article titles, authors, publication dates, volume/issue numbers
The dataset was constructed by scraping article metadata directly from the journal’s online archive, then cleaned and normalized for analysis.
I developed a Python script to systematically extract article metadata from the journal archive. The scraper navigates through each volume and issue, collecting titles, authors, and publication dates.
View the complete scraping script on GitHub →
Key challenges included: - Handling inconsistent formatting across different volumes - Extracting author names with various academic titles and credentials - Parsing dates from different formats
import pandas as pd
import matplotlib.pyplot as plt
from wordcloud import WordCloud
import re
from sklearn.feature_extraction.text import ENGLISH_STOP_WORDS
from collections import Counter
import seaborn as sns
from tabulate import tabulate
# Set visualization style
sns.set_palette("viridis")
plt.style.use('seaborn-v0_8-darkgrid')df = pd.read_csv("data/articles.csv")print(tabulate(df.head(), headers='keys', tablefmt='github', showindex=False))| title | authors | pages | date | article_url | issue_url |
|--------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|------------|--------------|------------------------------------------------------------------------|-------------------------------------------------------|
| FOREWORD | nan | p. 1-4 | October 1925 | https://onlinelibrary.wiley.com/doi/10.1111/j.1469-1809.1925.tb02036.x | https://onlinelibrary.wiley.com/toc/20501439/1925/1/1 |
| THE PROBLEM OF ALIEN IMMIGRATION INTO GREAT BRITAIN, ILLUSTRATED BY AN EXAMINATION OF RUSSIAN AND POLISH JEWISH CHILDREN | KARL PEARSON,, MARGARET MOUL, | p. 5-54 | October 1925 | https://onlinelibrary.wiley.com/doi/10.1111/j.1469-1809.1925.tb02037.x | https://onlinelibrary.wiley.com/toc/20501439/1925/1/1 |
| PROBLEM OF ALIEN IMMIGRATION | nan | p. 56-127 | October 1925 | https://onlinelibrary.wiley.com/doi/10.1111/j.1469-1809.1925.tb02038.x | https://onlinelibrary.wiley.com/toc/20501439/1925/1/2 |
| A PEDIGREE OF EPICANTHUS AND PTOSIS | C. H. USHER M.B., | p. 128-138 | October 1925 | https://onlinelibrary.wiley.com/doi/10.1111/j.1469-1809.1925.tb02039.x | https://onlinelibrary.wiley.com/toc/20501439/1925/1/2 |
| ON THE RELATIVE VALUE OF THE FACTORS WHICH INFLUENCE INFANT WELFARE. AN INQUIRY by ETHEL M. ELDERTON | Dr A. G. Anderson M.O.H.,, Dr Wm. Arnold Evans M.O.H.,, Dr Alfred Greenwood M.O.H.,, Dr H. O. Pilkington M.O.H.,, Dr C. H. Tattersall M.O.H., | p. 139-243 | October 1925 | https://onlinelibrary.wiley.com/doi/10.1111/j.1469-1809.1925.tb02040.x | https://onlinelibrary.wiley.com/toc/20501439/1925/1/2 |Converting publication dates to a standardized format for time-series analysis:
df["parsed_date"] = pd.to_datetime(df["date"], format="%B %Y", errors="coerce")
df["year"] = df["parsed_date"].dt.year
df["month"] = df["parsed_date"].dt.month
# Verify date parsing worked
print(f"Date range: {df['year'].min():.0f} to {df['year'].max():.0f}")
print(f"Total articles: {len(df)}")Date range: 1925 to 1953
Total articles: 523Preparing article titles for text analysis:
df["clean_title"] = (
df["title"]
.str.lower()
.str.replace(r"[^a-z0-9\s]", "", regex=True)
.str.replace(r"\s+", " ", regex=True)
.str.strip()
)How did the volume of publications change during the journal’s run?
year_counts = df["year"].value_counts().sort_index()
print("Articles published per year:")
print(year_counts)Articles published per year:
year
1925 6
1926 9
1927 6
1928 13
1930 6
1931 8
1933 17
1934 11
1935 18
1936 24
1937 17
1938 27
1939 32
1940 34
1941 41
1943 27
1946 38
1947 44
1949 38
1951 38
1952 39
1953 30
Name: count, dtype: int64plt.figure(figsize=(12, 6))
plt.plot(year_counts.index, year_counts.values, marker="o", linewidth=2, markersize=8)
plt.xlabel("Year", fontsize=12)
plt.ylabel("Number of Articles", fontsize=12)
plt.title("Publication Volume in Annals of Eugenics (1925-1954)", fontsize=14, fontweight='bold')
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
Key Findings: - Publication volume shows clear trends over the journal’s lifespan - Notable peaks and valleys may correspond to historical events (WWII, post-war period) - The journal maintained consistent output across most years
What themes dominated the research published in this journal?
# Combine all titles and remove stop words
all_titles = ' '.join(df['clean_title']).lower()
all_titles = re.sub(r'[^\w\s]', '', all_titles)
words = all_titles.split()
filtered_words = [word for word in words if word not in ENGLISH_STOP_WORDS]
word_counts = Counter(filtered_words)wordcloud = WordCloud(
width=1200,
height=600,
background_color='white',
colormap='viridis',
relative_scaling=0.5
).generate_from_frequencies(word_counts)
plt.figure(figsize=(16, 8))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.title("Most Frequent Terms in Article Titles", fontsize=16, fontweight='bold', pad=20)
plt.tight_layout()
plt.show()
top_words = word_counts.most_common(10)
df_top_words = pd.DataFrame(top_words, columns=['Word', 'Frequency'])
print("\nTop 10 Most Frequent Words in Titles:")
print(tabulate(df_top_words, headers='keys', tablefmt='github', showindex=False))
Top 10 Most Frequent Words in Titles:
| Word | Frequency |
|-------------|-------------|
| linkage | 40 |
| man | 32 |
| data | 32 |
| blood | 32 |
| inheritance | 26 |
| groups | 26 |
| analysis | 24 |
| note | 22 |
| study | 22 |
| reviews | 22 |plt.figure(figsize=(12, 6))
sns.barplot(x='Frequency', y='Word', data=df_top_words, palette='viridis')
plt.title('Top 10 Words in Article Titles', fontsize=14, fontweight='bold')
plt.xlabel('Frequency', fontsize=12)
plt.ylabel('Word', fontsize=12)
plt.tight_layout()
plt.show()/tmp/ipykernel_7604/3988971941.py:2: FutureWarning:
Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `y` variable to `hue` and set `legend=False` for the same effect.
sns.barplot(x='Frequency', y='Word', data=df_top_words, palette='viridis')
Key Findings: - Terms like “inheritance,” “genetics,” “correlation,” and “blood” dominate - Statistical and quantitative language is prominent (“frequency,” “data,” “analysis”) - Mix of biological/genetic terms and mathematical/statistical terminology reflects the journal’s interdisciplinary nature
Who were the most prolific contributors to the journal?
plt.figure(figsize=(12, 7))
sns.barplot(
x='count',
y='Author Name',
data=df_top_authors,
palette='magma'
)
plt.title('Top 15 Most Prolific Authors', fontsize=14, fontweight='bold')
plt.xlabel('Number of Publications', fontsize=12)
plt.ylabel('Author', fontsize=12)
plt.tight_layout()
plt.show()/tmp/ipykernel_7604/2290777036.py:2: FutureWarning:
Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `y` variable to `hue` and set `legend=False` for the same effect.
sns.barplot(
Key Findings: - Ronald Fisher was by far the most prolific contributor - Several other prominent statisticians and geneticists appear frequently (Haldane, Pearson, Penrose) - The top authors represent both British and international researchers - Notable presence of women researchers (Karn, Lawler, Holt) in an era when women faced significant barriers in science
The Annals of Eugenics was founded in 1925 by Karl Pearson and continued until 1954, when it was renamed the Annals of Human Genetics. This period spans:
The height of the eugenics movement (1920s-1930s)
World War II and the Holocaust (1939-1945)
Post-war reckoning with eugenics (late 1940s-1950s)
The name change in 1954 reflects the field’s shift away from eugenics toward human genetics as a legitimate scientific discipline.
This analysis reveals that while the journal bore the name “Eugenics,” much of the published research focused on statistical genetics, inheritance patterns, and quantitative methods. The prominence of terms like “correlation,” “inheritance,” and “blood groups” suggests a focus on empirical research rather than purely ideological work.
However, it’s important to note that:
The journal’s institutional context was explicitly eugenic
Even “objective” statistical research was often framed within eugenic ideology
The research published here was sometimes used to support discriminatory policies
Data Collection:
Web scraping may have missed some articles if they were formatted differently
The analysis only includes article metadata, not full text
Some author names may have been incorrectly normalized despite extensive cleaning
Analytical Scope:
This analysis focuses on publication patterns, not content or impact
Word frequency analysis of titles provides limited insight into actual research content
No assessment of citation patterns or influence
Historical Interpretation:
Quantitative analysis alone cannot capture the social and political context
The presence of certain authors or topics doesn’t indicate endorsement or criticism
This project demonstrates how data science methods can illuminate patterns in historical scientific publishing. By scraping, cleaning, and analyzing metadata from the Annals of Eugenics, we gain insight into:
This analysis forms part of a larger investigation into the history of eugenics in academia and provides a foundation for deeper historical research into this complex period.
All code and data for this project are available at:
github.com/ashleymikali/eugenics-journal-graphs
This project was completed as part of ongoing research into the history of eugenics at The College of Idaho and in Idaho more broadly.