Sentiment Intensity and Linguistic Toxicity in Political Discourse

2.1 · Collection

The original plan was to use official APIs from both platforms. However, X's developer API changed from a flat monthly rate to a usage-based model with strict daily pull limits, making this approach impractical without significant cost or time.

So rather than struggling with the official APIs, I turned to Apify, a third-party web scraping platform. It provides scrapers for both X and Bluesky with keyword filtering, feed type selection (top vs. latest), and structured JSON output.

I was able to skip the daily collection limits and building the dataset took hours instead of weeks of incremental pulls (queuing topics, monitoring output, and iterating on keyword filters to reduce off-topic noise). The final corpus spans approximately 19,000 posts across 24 collections (6 topics × 2 platforms × 2 feed types), with roughly 800 posts per cell.

2.2 · Preprocessing

The raw JSON output from Apify contained far more than post text: full user objects, engagement counts, embedded media metadata, reply chains, and retweet wrappers. The first pass stripped everything down to: post text, platform, feed type, topic, and a timestamp. Retweets (posts beginning with "RT @") were dropped as they could have inflated counts for whichever accounts happened to go viral that week. Very short posts (under five tokens after cleaning) were also removed, as they might not carry enough information for the sentiment or toxicity models.

There was also a need to apply keyword filters. The ICE enforcement query returned a large volume of off-topic content like "ice sculpture," "ice cream," "ice cold," hockey game recaps, and weather reports about ice storms. These were filtered by requiring additional terms like "immigration," "deportation," "border," or "enforcement." The job market topic had a similar problem where actual job listings ("now hiring," "apply today") and LinkedIn-style promotional posts were included. These were filtered out by removing posts with application URLs and common recruiter phrasing. Tariffs and Israel-Gaza were cleaner where off-topic noise was low. Coffee and weather were intentionally broad as the goal was to capture casual, low-stakes conversation.

After filtering, URLs were stripped, whitespace normalized, and each post tagged with its source platform, feed type, and topic.

2.3 · Corpus composition

3.1 · Sentiment analysis

The two most common alternatives for social media sentiment, VADER and TextBlob, were both considered and rejected. Prior research has applied both tools to social media toxicity tasks (Singgalen, 2024), demonstrating their utility in controlled settings but also their sensitivity to domain mismatch. I chose DistilBERT (Sanh et al., 2019) fine-tuned on the Stanford Sentiment Treebank (Socher et al., 2013) because the softmax output gives a per-class probability and a confidence score that neither VADER nor TextBlob had. This confidence gives us the signed intensity measure: a post classified NEGATIVE with 0.97 confidence is treated differently from one classified NEGATIVE with 0.52 confidence, and that distinction is central to the platform comparison in Section 4.1. What all three models do not do, and what no off-the-shelf model reliably detects, is sarcasm. This is a limitation discussed in Section 5.3.

I also conducted a test to see if the models could detect sarcasm. Each post receives a binary label (POSITIVE / NEGATIVE) and a confidence score in [0.5, 1.0]. I constructed a signed sentiment score by assigning negative posts a negative value, mapping the full range to [−1, 1], capturing both direction and intensity in a single measure.

3.2 · Toxicity analysis

The main alternative considered was the Perspective API from Google. Beyond the problems of using an authenticated API call per post which is infeasible for 19,000 posts in an offline pipeline, Google has announced that Perspective API is being sunset after 2026. I also looked at HateSonar, but it only outputs a single binary hate/not-hate label with no subtype breakdown.

Detoxify's Toxic-BERT (Hanu & Unitary team, 2020) runs fully locally and was trained on the Jigsaw Toxic Comment Classification which is a large, well-documented dataset of online comments covering a wide range of harmful language styles. It outputs six distinct scores per post rather than a single toxicity flag: overall toxicity, severe toxicity, obscenity, insult, threat, and identity attack. That subtype ended up being useful for the high-toxicity slice analysis in Section 4.6 shows that most extreme content is driven by obscenity and insults rather than threats, which would have been invisible with a single-score model.

3.3 · Statistical testing

Three tests were considered for platform comparisons: Student's t-test, Welch's t-test, and Mann-Whitney U (Mann & Whitney, 1947). The t-test has a practical problem with this data. The toxicity distributions are heavily right-skewed where over 90% of posts score below 0.05, with a long tail reaching near 1.0. The t-test operates on means, which are pulled upward by that tail. A single viral high-toxicity post can shift the platform mean enough to affect a p-value, without any real difference in how the typical post on each platform behaves. Mann-Whitney U avoids this entirely by ranking all observations so an outlier at 0.99 receives one rank instead of 100× the weight of a post at 0.01.

The effect size is reported as rank-biserial r = 1 − (2U) / (n₁ · n₂). A positive r means X posts tend to rank higher in toxicity than Bluesky posts; negative means the reverse. The magnitude scale is: |r| < 0.1 = negligible, 0.1–0.3 = small, 0.3–0.5 = medium, > 0.5 = large. The rank-biserial r is bounded to [−1, 1] and has a direct probabilistic interpretation where r = 0.3 meaning a randomly drawn X post ranks above a randomly drawn Bluesky post 65% of the time.

4.1 · Sentiment overview

Across all topics, political content is overwhelmingly negative. Tariffs had the lowest rate of positive posts (16.8%), followed by ICE enforcement (19.3%) and Israel-Gaza (21.9%). The two control topics were markedly more positive: coffee (41.0%) and weather (44.7%).

4.2 · Platform differences in sentiment

The largest sentiment gap between platforms appears on the Israel-Gaza topic where Bluesky users post positively only 15.4% of the time, compared to 28.4% on X (13% difference). On coffee and weather (the controls), platforms are essentially identical (≈41% and ≈44–45% positive respectively), confirming the gap is content-driven rather than a platform artifact.

Signed sentiment intensity (mean confidence mapped to [−1, 1]), where more negative values indicate more intensely negative posts reveals a consistent pattern where all six topics trend negative on both platforms, but political topics are far more intensely negative. The table below reports mean negativity intensity as a percentage for each topic and platform.

4.3 · Toxicity by platform and topic

The central finding of this study is that X is not uniformly more toxic than Bluesky. The direction of the difference depends on the topic. The table below shows mean toxicity scores disaggregated by platform and topic.

Control topics show near-equal toxicity across platforms, confirming that observed differences on political topics reflect real discourse differences rather than platform-level measurement artifacts.

4.4 · Statistical significance and effect sizes

Mann-Whitney U tests were run for each topic comparing X and Bluesky. Five of six topics show a statistically significant toxicity difference. The single exception is Job Market (p = 0.207), where the platform gap in means does not hold at the rank level suggesting the mean difference is driven by outliers rather than a systematic shift. ICE is the only topic with a medium effect size (|r| = 0.30); all others are small.

For signed sentiment intensity, three topics show significant platform differences where Tariffs, Job Market, and Israel all in the direction of X being more positive (less intensely negative) than Bluesky, consistent with Bluesky's more activist user base on economic and geopolitical topics.

4.5 · Feed type: latest vs. top

Algorithmically promoted (top) feeds are consistently less toxic than chronological (latest) feeds on politically charged topics. The table below shows this pattern holds for ICE, Israel-Gaza, and Tariffs, but not for the apolitical coffee control.

4.6 · Toxicity subtypes

In the high-toxicity slice (posts above the 95th percentile, threshold ≈ 0.43), the dominant subtypes are obscenity (mean score 0.46) and insult (0.35), with threats (0.05) and severe toxicity (0.05) being comparatively rare. Within this slice for the ICE topic, X accounts for 116 of 150 posts (3.4:1 ratio relative to Bluesky's 34) despite the two platforms contributing roughly equal post counts overall.

This shows that extreme content in political discourse is primarily driven by crude language and personal insults rather than explicit threats. Identity attacks are concentrated on immigration and geopolitical topics (ICE, Israel) which is consistent with the toxicity gap between platforms on those subjects.

5.1 · Platform effect is topic-conditional

The assumption that X is more toxic than Bluesky is only partially supported. For topics involving immigration enforcement and geopolitical conflict where discourse on X skews toward outrage and engagement-driven content where X is significantly more toxic. For economic topics like tariffs and job loss, Bluesky's discourse is equally or more charged. This may reflect the platform's early user base of tech workers and journalists who feel economicly inclined and express it with comparable intensity.

The finding that Bluesky users express more intensely negative sentiment on Israel, Job Market, and Tariffs, despite lower toxicity scores, suggests that negativity and toxicity can be dissociable constructs. This means that it is possible to hold and express strong negative opinions without producing content that scores high on toxic language measures.

5.2 · What algorithmic curation does

The consistent reduction in toxicity between latest and top feeds (Section 4.5) is noteworthy and somewhat counterintuitive. If confirmed across a larger sample, it suggests that recommendation algorithms, often blamed for amplifying divisive content, may in practice filter out the most harmful posts from prominent placement, even if they also boost engagement through other mechanisms. The fact that this dampening effect does not appear on control topics (coffee) is particularly informative as it implies the algorithm is not merely selecting for popularity since popular coffee posts are no less toxic than unpopular ones.

5.3 · Limitations

Several limitations qualify these findings.

• Model domain gap. The sentiment model (DistilBERT fine-tuned on SST-2, a movie review dataset) may misread political sarcasm, irony, or domain-specific terminology. The sarcasm failure demonstrated in Table 3, where all three models classify "deportation flights are great, actually" as strongly POSITIVE, illustrates this point.
• Toxicity training data. Toxic-BERT was trained on Wikipedia talk page comments and Jigsaw competition data, which differs in register from social media text. It may under-flag implicit toxicity common in political social media (veiled dehumanization, dog whistles) while correctly flagging explicit profanity.
• Sample size per cell. Each collection cell was capped at ≈ 800 posts, limiting statistical power for subgroup analyses (e.g., feed type within platform within topic).
• No demographic data. Observed platform differences may partly reflect user composition rather than platform norms per se. Bluesky's early user base was not representative of the general population; the platform has diversified since its public launch but demographic data is not available at the post level.
• Temporal snapshot. Data was collected during a specific window in early 2026. Discourse around ICE enforcement and Israel-Gaza may be particularly time-sensitive; findings may not generalize to other periods.

5.4 · LLM-based sarcasm audit

Section 5.3 identifies sarcasm detection as a known failure mode of DistilBERT. To quantify how often this failure actually produces a mislabeled post in practice, I conducted a targeted validation using a local large language model: llama3.2, run via Ollama on the same machine as the inference pipeline, requiring no external API calls and producing fully reproducible results.