Can AI & Machine Learning Beat Kalshi...or Is It Just Luck?

Summary

The experiment explores using machine learning to predict 15-minute Bitcoin Up or Down contracts on Kalshi. The dataset comprises 5,335 historical resolved markets collected across 58 days, with 606 untouched markets reserved as an out-of-sample test set. The model observes five one-minute candles during the first five minutes of the market window (at the t minus 600 mark) and attempts to predict the outcome ten minutes ahead before market resolution at t zero. In out-of-sample testing, the model achieved a log loss of 0.516982 compared to Kalshi's market log loss of 0.518414, reflecting a marginal edge of 0.001432. This small difference shows that Kalshi market prices already incorporate almost all available information.

Despite the tiny theoretical edge, an initial model run correctly predicted seven consecutive live market directions. Based on this outcome, an automated bot written in Python was deployed to execute live trades with one-dollar stakes. Over sixteen one-dollar live trade windows, the bot recorded twelve wins and four losses, delivering a seventy-five percent directional win rate. Encouraged by these early returns, the stake was raised to approximately five dollars per trade to observe performance under higher capital allocation, starting with a portfolio cash balance of 56.62 dollars.

During live execution with five-dollar stakes, the bot traded consecutive fifteen-minute intervals, taking either Yes or No positions depending on model probabilities. The account balance climbed steadily through multiple wins, surpassing 65.82 dollars, 73.79 dollars, and eventually reaching 88.12 dollars. Across the five-dollar live trading run, the system finalized fourteen trades, achieving thirteen wins and one loss. This produced a 92.86 percent win rate and a net profit of 31.50 dollars after transaction fees, yielding a portfolio return of over 174 percent in approximately three hours.

A subsequent quantitative post-mortem conducted using an AI coding assistant and Python audited the live results. The analysis revealed that the model and Kalshi took identical directional sides across all fourteen markets, with the model adjusting Kalshi's implied probabilities by an average of only 1.03 cents. Statistical projection over a simulation of 1,000 trade windows indicated that the win rate would regress to approximately 70 percent, producing an expected net loss of roughly twelve dollars due to taker fees averaging 1.62 cents per contract against an estimated gross model edge of 1.48 cents. The repository containing data extraction guidelines, LLM prompts, requirements, and training code using LightGBM and XGBoost was published on GitHub for educational experimentation.