aether_quant.data¶
Parsers and tables of the data pipeline (fetchers and backfills stay in data_pipeline/).
Parsers and tables of the data pipeline (fetchers and backfills stay in data_pipeline/).
adaptive_sell_threshold ¶
adaptive_sell_threshold(*, sell_threshold: float, history: list[float] | None, band_enabled: bool, min_observations: int, band_percentile: float) -> float
Pure lookup + percentile over accumulated probability_up history (main.py::_effective_sell_threshold verbatim). Never adapts ABOVE the static threshold - the band only makes selling easier.
bars_to_panel ¶
bars_to_panel(bars: dict[str, DataFrame], field: str = 'close') -> pd.DataFrame
{ticker: bars} to a dates x tickers frame of one field, indexed by YYYY-MM-DD strings, NaN where a coin has no bar.
base_asset ¶
base_asset(symbol: str) -> str | None
LUNAUSDT -> LUNA. None for a pair this phase doesn't use: not quoted in USDT, a stablecoin, a leveraged token.
candle_windows ¶
candle_windows(start: date, end: date, max_rows: int = MAX_ROWS) -> list[tuple[date, date]]
Inclusive [start, end] cut into windows of at most max_rows - 1 days, oldest first (one spare row of headroom).
candles_url ¶
candles_url(product: str, start: date, end: date) -> str
The Coinbase public candles URL for one product and an inclusive day window, daily granularity.
canonical_issuer ¶
canonical_issuer(ticker: str) -> str
The issuer a ticker belongs to, through renames and share-class twins, so one company counts once.
change_by_session ¶
change_by_session(positioning: DataFrame, sessions: list[str], weeks: int = CHANGE_WEEKS) -> pd.DataFrame
session x market: the weeks-report change in net positioning, as known at each session's close. A report
published on date P (after the close) is usable on the first session after P.
coinbase_product ¶
coinbase_product(ticker: str) -> str
BTCUSD -> BTC-USD; a ticker that already has a dash is returned as it is.
collect_cached_eps ¶
collect_cached_eps(tickers: Iterable[str], cache_dir: Path) -> list[dict]
The quarterly EPS rows fetch_fundamentals saved, in read_eps's shape: the same download serves both.
count_cosine ¶
count_cosine(left: Mapping[int, int], right: Mapping[int, int]) -> float | None
The cosine of two count vectors, or None when either is empty.
coverage_by_year ¶
coverage_by_year(table: Mapping[str, list[tuple[str, str]]], available: set[str], first_year: int = 2004, last_year: int = 2026) -> dict[str, float]
Share of member-days per year on names with no usable bars. Anything left out of available, for whatever reason, counts.
daily_funding ¶
daily_funding(settlements: list[tuple[str, float]]) -> dict[str, float]
A day's funding is the sum of its settlements: what a long position pays that day.
default_get_json ¶
default_get_json(url: str, *, timeout: float = 20.0, retries: int = 3, backoff: float = 1.5) -> object
GET a JSON document with the standard library. Retries a transient failure with exponential backoff.
derive_signal ¶
derive_signal(probability_up: float, *, decision_threshold: float, buy_threshold: float, effective_sell_threshold: float, max_position_weight: float, record_observation: Callable[[str | None, float], None] | None = None, symbol_key: str | None = None) -> tuple[str, float, float]
Signal classification extracted from main.py::_derive_signal()
(verbatim arithmetic). record_observation(symbol_key, probability_up)
is an optional callback so callers can preserve the observation-recording
side effect without this function needing state.
extended_rows ¶
extended_rows(facts: dict[str, Any]) -> list[dict]
Every V6.6.1 field. For a field with several tags, an earlier tag wins a fiscal year, as in fundamentals_rows.
fetch_daily_bars ¶
fetch_daily_bars(tickers: Sequence[str], start: date, end: date, *, get_json: GetJson = default_get_json, cache_dir: Path | None = DEFAULT_CACHE_DIR, today: date | None = None) -> tuple[dict[str, pd.DataFrame], list[str]]
({ticker: bars}, failed tickers). Bars cover [start, min(end, yesterday)] and never include today's forming bar. A cache written for an earlier-or-equal start and a later-or-equal end is reused without a call.
filing_earnings_rows ¶
filing_earnings_rows(filings: Sequence[Mapping]) -> list[dict]
8-Ks that carry Item 2.02 (results of operations): accepted is the acceptance timestamp, filed the filing date.
filing_similarity_rows ¶
filing_similarity_rows(filings: Sequence[Mapping], counts_of: Callable[[Mapping[str, Any]], Mapping[str, int] | None]) -> list[dict]
One row per filing that has a year-earlier twin: ticker-free {form, period_end, filed, similarity}. counts_of(row) returns that filing's hashed counts (or None when it can't be read).
filing_tokens ¶
filing_tokens(html: str) -> list[str]
Lower-case words of three letters or more, with tags and entities removed.
first_filings_annual ¶
first_filings_annual(facts: dict[str, Any], tag: str, *, taxonomy: str = 'us-gaap') -> list[dict]
{period_end, filed, form, value} per fiscal year, first filing only, from annual forms: what was public, not what a later restatement says. A figure may appear in many later filings as a comparative; only the earliest counts.
flatten_fill_forward_bar ¶
flatten_fill_forward_bar(bar: dict[str, Any], previous_close: float) -> dict[str, Any]
A fill-forward bar the way the training dataset builds it (train.fill_forward_forex_frame): flat at the
previous close with no volume. Lean's own fill-forward bar repeats the previous bar's range, so a trailing ATR or
high-low spread window that contains one differs from the dataset's (V6.0.0, todo lead 4).
fundamentals_rows ¶
fundamentals_rows(facts: dict[str, Any]) -> list[dict]
Every field's annual first filings. For a field with several tags, an earlier tag wins a fiscal year; a later one only fills years it doesn't have.
fundamentals_table ¶
fundamentals_table(rows: DataFrame, sessions: list[str], columns: list[str], field: str, *, lag_sessions: int = 1) -> pd.DataFrame
Session x ticker, each cell the latest fiscal year's value filed by then. A filing dated D is usable a session later: it can arrive after the close. Nothing filed later shows, whatever fiscal year it covers.
funding_url ¶
funding_url(symbol: str, month: str) -> str
The data.binance.vision URL of one symbol's monthly funding-rate archive.
fx_pair_scores ¶
fx_pair_scores(change: DataFrame, pairs: Iterable[str]) -> pd.DataFrame
Each pair's score: the change in speculators' positioning in its base currency minus that in its quote currency (the dollar has no contract of its own and counts as zero).
hashed_counts ¶
hashed_counts(words: Sequence[str], buckets: int = BUCKETS) -> dict[int, int]
Word counts keyed by a stable hash (blake2b, so it doesn't change between runs or machines).
insider_net_table ¶
insider_net_table(trades: DataFrame, sessions: list[str], columns: list[str], *, window_days: int = 180, lag_sessions: int = 1) -> pd.DataFrame
Session x ticker: open-market buys less sales over the trailing window_days, by filing date, a session late: a filing can
arrive after the close. Nothing in the window is NaN for a covered ticker: no insider trading is a real 0.
is_member ¶
is_member(table: Mapping[str, list[tuple[str, str]]], ticker: str, date: str) -> bool
Inside one of the ticker's intervals on date; the end date itself is out.
kline_url ¶
kline_url(symbol: str, month: str) -> str
The data.binance.vision URL of one symbol's monthly daily-kline archive.
liquid_universe ¶
liquid_universe(quote_volume: DataFrame, *, top_n: int = 30, window: int = 60) -> pd.DataFrame
Date x coin, True where the coin is among the top_n by trailing dollar volume. Trailing and shifted a row: a day's own
volume isn't known until it ends. Only a coin with a full window qualifies.
list_symbols ¶
list_symbols(get_text: Callable[[str], str], prefix: str = KLINE_PREFIX) -> list[str]
Every folder under prefix, following the listing's pages: it returns 1,000 entries at a time, and stopping at the first
page would silently lose most of the alphabet.
listed_months ¶
listed_months(get_text: Callable[[str], str], prefix: str) -> list[str] | None
The YYYY-MM of every file under prefix, or None if the listing holds none: then the caller tries every month. A pair has files only for
the months it traded, so asking first saves most of the requests.
load_membership ¶
load_membership(path: Path) -> dict[str, list[tuple[str, str]]]
Read the S&P 500 membership file into {ticker: [(start, end), ...]}.
macro_scores ¶
macro_scores(change: DataFrame) -> pd.DataFrame
Direction for the index and the bond ETFs from the S&P and 10-year contracts.
market_cap_table ¶
market_cap_table(raw_closes: DataFrame, rows: DataFrame, splits: DataFrame, *, lag_sessions: int = 1) -> pd.DataFrame
Close x shares outstanding, as of each session. raw_closes are Yahoo's split-adjusted closes without dividends, so a count
as filed is restated by the splits after its filing date to match: the product can't depend on any later split or dividend. An
adjusted close would embed later dividends, and valuation ratios built on it would read the future.
member_days_by_year ¶
member_days_by_year(table: Mapping[str, list[tuple[str, str]]], tickers: Iterable[str], first_year: int, last_year: int) -> dict[int, float]
Calendar member-days per year for tickers.
merge_submission_pages ¶
merge_submission_pages(pages: Sequence[Mapping]) -> list[dict]
Filing rows from the recent block and any older pages (SEC's filings.files), oldest first, one row per accession.
midpoint_bar_from_quote_bar ¶
midpoint_bar_from_quote_bar(quote_bar: Any) -> SimpleNamespace | None
V4.6 - forex quote-bar (bid/ask) to trade-bar-shaped midpoint OHLC (main.py::_midpoint_bar_from_quote_bar verbatim). Volume is always 0.0 - forex quote data carries no trade-volume concept. Returns None (never raises) when no quote bar exists or it's missing a bid/ask side.
months_between ¶
months_between(start: str, end: str) -> list[str]
YYYY-MM from start to end, inclusive.
opportunistic ¶
opportunistic(trades: DataFrame) -> pd.DataFrame
The trades of insiders not routine at that moment: routine means a trade in this calendar month of each of the three prior years, which uses nothing after the trade. An insider with too little history is opportunistic, as in the paper.
pad_sequence_history ¶
pad_sequence_history(history: list[list[float]], window_size: int) -> list[list[float]]
Left-pads a rolling per-symbol feature-history buffer with zero
vectors up to window_size (main.py::_pad_sequence_history verbatim).
Pure, no side effects. An empty history returns window_size all-zero
vectors of the model's width when inferable, else raises a clear
ValueError instead of IndexError (V5.4.7 #123).
parse_annual_text ¶
parse_annual_text(text: str) -> list[dict]
Rows of one annual.txt, for the markets above: {report_date, publication_date, market, net_pct_oi}. A row with a
missing or zero open interest is dropped.
parse_aqr_sheet ¶
parse_aqr_sheet(payload: bytes, *, sheet: str, name: str, column: str = 'USA') -> pd.Series
One factor's daily return from an AQR workbook, whose table starts below some prose. Dates are MM/DD/YYYY or real dates.
parse_candles ¶
parse_candles(rows: object) -> pd.DataFrame
Rows to a frame with columns date, open, high, low, close, volume, ascending by date. Drops malformed, non-finite,
non-positive-price and duplicate rows (the first of a duplicated date is kept).
parse_day ¶
parse_day(text: str, wanted: dict[str, str] | None = None) -> dict[str, float]
{ticker: short volume / total volume}. A file ends with a record count, and a row with no volume says nothing.
parse_french_zip ¶
parse_french_zip(payload: bytes) -> pd.DataFrame
Decimal returns by YYYY-MM-DD. The file has prose before its header and after its data; only dated rows are kept. The library states returns in percent and marks a missing value with -99.99 or -999.
parse_funding_zip ¶
parse_funding_zip(payload: bytes) -> list[tuple[str, float]]
(date, rate) per settlement from one monthly file: calc_time,funding_interval_hours,last_funding_rate.
parse_kline_zip ¶
parse_kline_zip(payload: bytes) -> list[dict]
{date, open, high, low, close, volume, quote_volume} from one monthly file. Some files have a header row and some don't.
parse_membership ¶
parse_membership(text: str) -> dict[str, list[tuple[str, str]]]
{ticker: [(start, end), ...]}, renames merged, sorted. end is exclusive and far in the future for a current member.
parse_quarter ¶
parse_quarter(payload: bytes, wanted: set[str] | None = None) -> pd.DataFrame
TRADE_COLUMNS rows of one quarter's zip: non-derivative open-market trades with a price.
parse_zip_bytes ¶
parse_zip_bytes(payload: bytes) -> list[dict]
Rows of the first text file in a CFTC Commitments of Traders zip, or [] when it holds none.
price_breaks ¶
price_breaks(closes: Mapping[str, float], *, ratio: float = 10.0) -> list[dict]
Consecutive closes ratio times apart or more, across any gap in the dates: a redenomination or token swap spliced into one pair
(COCOS went from 0.0003 to 0.7 over a three-week gap in 2021), which reads as a return nobody could earn. A listing's first days can
do this honestly too, so a break is a thing to look at, not proof.
prior_year_filing_pairs ¶
prior_year_filing_pairs(filings: Sequence[Mapping]) -> list[tuple[Mapping, Mapping]]
(this filing, the same form a year earlier) for each 10-K and 10-Q that has one, matched on the report date: 330 to 400 days apart, the nearest to a year. Never a later filing.
quarterly_first_filings ¶
quarterly_first_filings(facts: dict[str, Any]) -> list[dict]
{period_start, period_end, filed, eps} for every standalone quarter, first filing only. Year-to-date and annual figures are dropped by their duration; the first tag with data wins.
quarterly_ttm_rows ¶
quarterly_ttm_rows(facts: dict[str, Any]) -> list[dict]
ttm_net_income, ttm_operating_cash_flow and assets_q, in FUNDAMENTALS_COLUMNS' shape without the ticker.
quarters ¶
quarters(start_year: int, end_year: int, end_quarter: int) -> list[tuple[int, int]]
Every (year, quarter) from start_year Q1 to end_year Qend_quarter, oldest first.
read_eps ¶
read_eps(path: Path = OUTPUT) -> pd.DataFrame
Read the cached EDGAR earnings-per-share table, with its date columns as strings.
read_fundamentals ¶
read_fundamentals(path: Path) -> pd.DataFrame
Read a cached EDGAR fundamentals table, with its keys and dates as strings.
read_positioning ¶
read_positioning(path: Path = OUTPUT) -> pd.DataFrame
Read the cached CFTC positioning table, with its dates and market as strings.
read_splits ¶
read_splits(path: Path = SPLITS_OUTPUT) -> pd.DataFrame
Read the cached stock-split table, with ticker and date as strings.
sector_for_sic ¶
sector_for_sic(sic: int | str | None) -> str
A sector label from a SEC SIC code, in the GICS sector names. Coarse, and today's code rather than the code at each date: it
only sets which names are ranked together. A company with no code is unknown, which is never traded.
short_ratio_table ¶
short_ratio_table(ratios: DataFrame, sessions: list[str], columns: list[str], *, window: int = 20, lag_sessions: int = 1) -> pd.DataFrame
Session x ticker: the mean daily short ratio over window sessions, a session late, since the file posts in the evening. A gap
stays NaN: a file that never arrived isn't a ratio of zero.
split_adjusted_eps ¶
split_adjusted_eps(eps: DataFrame, splits: DataFrame) -> pd.DataFrame
EPS on one share basis per ticker. A first filing states EPS on the shares at its filing date, so it is divided by the splits after that date. A later split scales a ticker's whole history alike, so a standardised ratio sees no future.
split_factor_after ¶
split_factor_after(splits: DataFrame, ticker: str, filed: str) -> float
Product of the ticker's split ratios after filed, up to today: what turns a share count as filed into Yahoo's units.
standardised_surprises ¶
standardised_surprises(group: DataFrame) -> np.ndarray
One ticker's rows sorted by period end: the change from the same quarter a year earlier, over the spread of the latest earlier changes. Every input was filed by this row's filing date.
surprise_table ¶
surprise_table(eps: DataFrame, sessions: list[str], columns: list[str], *, hold_sessions: int) -> pd.DataFrame
session x ticker: standardised_surprises of the latest filing, from the session after it was filed for
hold_sessions sessions; NaN otherwise, so a ticker is ranked only while its news is recent. Pass split-adjusted EPS.
suspect_interval ¶
suspect_interval(intervals: Iterable[tuple[str, str]], first_bar: str, *, window_start: str, grace_days: int = 30) -> bool
True when the data begins well after a member interval inside the window began: a later company reusing the ticker, or a rename missing from RENAMES. Its bars are not this member's, so the name is left out rather than trusted.
tickers_touching ¶
tickers_touching(table: Mapping[str, list[tuple[str, str]]], start: str, end: str) -> list[str]
Members at any point of [start, end].
truncate_at_swap ¶
truncate_at_swap(rows: list[dict], *, ratio: float = 10.0, min_gap_days: int = 2) -> tuple[list[dict], dict | None]
A pair's bars up to its first break across a gap of min_gap_days or more: what follows is another instrument under the same name. A collapse with
no gap (LUNA on 2022-05-11) is a real loss and stays.
user_agent ¶
user_agent() -> str
SEC refuses a client with no contact. The address comes from the environment so that it never enters the repository.
write_fundamentals ¶
write_fundamentals(rows: list[dict], path: Path) -> None
Write fundamentals rows as a gzipped CSV in the canonical column order, creating the folder.
yahoo_symbol_for_member ¶
yahoo_symbol_for_member(ticker: str) -> str
BRK.B on Lean and in the membership list is BRK-B on Yahoo.