# Replication package (MATLAB)

This folder contains code and pre-generated outputs for the paper:
`INDIVIDUAL AND COMMON INFORMATION: MODEL-FREE EVIDENCE FROM PROBABILITY FORECASTS`
by Yizhou Kuang and Kristoffer Nimark

## Directory map

- `Data_clean_and_info_measure/`
	- SPF micro data cleaning, signal extraction, and construction of
	information measures.
	- Main packed output:
	`Data_clean_and_info_measure/output/info_measures.mat`
	(struct `info`).
- `Stylized_facts/`
	- Uses `info_measures.mat` plus macro series (CPI,
	unemployment, VIX, ANX, NBER recession indicator) to generate paper
	figures and the correlation tables.
- `Heatmap/`
	- Generates heatmaps of average SPF probability forecasts and bin
	boundaries.

## MATLAB requirements

- MATLAB with basic statistics and plotting functionality (uses
`corr`, `readtable`, figures).
- Reading Excel `.xls/.xlsx` files must work on your system
(on some Linux setups you may need to convert to CSV).

Formatting-only note: -
`Stylized_facts/main_stylized_facts.m` calls
`fontsize(...)` and `setprinttemplate(...)`. If
these are missing on your path, replace with standard
`set(gca,'FontSize',...)` etc.; numerical results are
unchanged.

## Recommended run order (from scratch, you can just run each main
without problem if you don’t want to.)

1. **Clean the SPF probability forecasts** (creates
`*_cleaned.mat` in
`Data_clean_and_info_measure/output/`)
	- Run the five scripts:
		- `Data_clean_and_info_measure/clean_gdpoutput.m`
		- `Data_clean_and_info_measure/clean_gdpprice.m`
		- `Data_clean_and_info_measure/clean_pce.m`
		- `Data_clean_and_info_measure/clean_cpi.m`
		- `Data_clean_and_info_measure/clean_unemp.m`
2. **Extract common and individual signals**
	- Run:
	`Data_clean_and_info_measure/main_signal_decomposition.m`
	- Output:
	`Data_clean_and_info_measure/output/signals_*.mat`
3. **Compute information measures and pack everything**
	- Run:
	`Data_clean_and_info_measure/main_info_measures.m`
	- Output:
		- `Data_clean_and_info_measure/output/info_measures.mat`
		(struct `info`)
		- plus per-variable `.fig` files in the same folder
		(`update_meas_*.fig`, `neg_entropy_meas_*.fig`,
		`Precision_*.fig`, `mean_*.fig`, histogram
		figures).
4. **Stylized facts + correlation tables**
	- Run: `Stylized_facts/main_stylized_facts.m`
	- Inputs:
		- `../Data_clean_and_info_measure/output/info_measures.mat`
		- `Stylized_facts/Data/macro_variables_data_raw/*.xls`
	- Outputs:
		- paper-ready PDFs in `Stylized_facts/` (listed below)
		- correlation matrices and p-values in the MATLAB workspace
		(`CORR_*` and `PVAL_*`).
5. **Heatmaps**
	- Run: `Heatmap/heatmap_main.m`
	- Output PDFs/PNG in `Heatmap/`.

## Where to find each paper result

### Figure 2.1 (heatmaps of average probability forecasts + bin
boundaries)

- Code: `Heatmap/heatmap_main.m`
- Outputs: `Heatmap/combined_heatmap.pdf` (and
`.png`), plus `Heatmap/bin_heatmap_*.pdf`.

### Figure 2.2, 3.1, 4.1, 4.2 (illustrations)

- Schematic embedded in the PDF (not produced by MATLAB).

### Figure 5.1 (time series of informativeness measures)

- Code: `Stylized_facts/main_stylized_facts.m`
- Outputs:
	- `Stylized_facts/measures_9by1.pdf` (main multi-panel
	figure)
	- `Stylized_facts/measures_CPI.pdf`,
	`Stylized_facts/measures_unemp.pdf`,
	`Stylized_facts/measures_GDP.pdf`.
- Underlying per-variable MATLAB figures are also in
`Data_clean_and_info_measure/output/` as `.fig`
files:
	- `update_meas_*.fig`, `neg_entropy_meas_*.fig`,
	`Precision_*.fig`.

### Table 1 (CPI inflation correlations; full sample and COVID-free
panel)

- Code: `Stylized_facts/main_stylized_facts.m`, turn the
nob `COVID_FREE_CORR` to true for covid-free results
- Workspace variables:
	- `CORR_CPI` and `PVAL_CPI`.
- Significance levels: p-values come from MATLAB
`corr(...,'rows','complete')`.

### Table 2 (Unemployment correlations; full sample and COVID-free
panel)

- Code: `Stylized_facts/main_stylized_facts.m`
- Workspace variables:
	- `CORR_UNEMP` and `PVAL_UNEMP`.

### Table 3 (Philadelphia Fed Anxious Index correlations)

- Code: `Stylized_facts/main_stylized_facts.m`
- Input series:
`Stylized_facts/Data/macro_variables_data_raw/ANX_1969q1.xls`
- Workspace variables:
	- `CORR_ANX` and `PVAL_ANX`.

### Table 4 (VIX correlations)

- Code: `Stylized_facts/main_stylized_facts.m`
- Input series:
`Stylized_facts/Data/macro_variables_data_raw/VIX_1990q1.xls`
- Workspace variables:
	- `CORR_VIX` and `PVAL_VIX`.

### NBER recession correlations (mentioned as available in replication
files)

- Code: `Stylized_facts/main_stylized_facts.m` (search for
`NBER`)
- Outputs in workspace (variables named like
`corr_*_NBER_*` and `pval_*_NBER_*`).

### Supplementary / appendix figures produced by the code

Generated by `Stylized_facts/main_stylized_facts.m`: -
`Stylized_facts/hetero_CPI.pdf` -
`Stylized_facts/x_sec_measures_3x3.pdf` (and
`.eps`) - `Stylized_facts/x_sec_corrs_3x3.pdf`
(and `.eps`).

### Other results (not shown as tables/figures)

#### Footnote 11 — Longer-horizon panels

Longer-horizon versions (next year / year-after-next) are included in the replication outputs.

* **Data:** `Data_clean_and_info_measure/output/info_measures.mat`
  (`T × Q` arrays for `VAR ∈ {GDP, P_GDP, PCE, CPI, unemp}`)

  * `info.series.(VAR).KL_pub`, `info.series.(VAR).KL_priv`
  * `info.series.(VAR).neg_H_pub`, `info.series.(VAR).neg_H_priv`
  * `info.series.(VAR).mean_pub`, `info.series.(VAR).mean_pr`
  * `info.series.(VAR).prec_pub`, `info.series.(VAR).prec_pr`
* **Figures:** `Data_clean_and_info_measure/output/*.fig` (multi-panel when `Q>7`)

  * `update_meas_<VAR>.fig`, `neg_entropy_meas_<VAR>.fig`, `mean_<VAR>.fig`, `Precision_<VAR>.fig`

#### Footnote 13 — NBER recession correlations

Correlations between information measures and the NBER recession indicator.

* **Code:** `Stylized_facts/main_stylized_facts.m` (search `NBER`)
* **Outputs:** MATLAB workspace variables named like `corr_*_NBER_<VAR>` and `pval_*_NBER_<VAR>`.

#### Illustration 1 — with increased signal precision but more uncertain

This block constructs and plots two diagnostic series (not used as a paper table/figure). 

* **Code:** `Stylized_facts/main_stylized_facts.m` (search `Construct two new measures`). Results are shown in Figure 13 and Figure 14
* **Definitions (as implemented):**

  * `A_t` (`stats_1_<VAR>`): indicator that
    (i) average individual precision change `mean_i Δprec_ind_{i,t} ≥ 0`,
    (ii) common precision change `Δprec_common_t ≥ 0`, and
    (iii) variance change `Δvar_t ≥ 0`.
  * `B_t` (`stats_2_<VAR>`): fraction of agents satisfying the same sign conditions,
    `B_t = (1/J_t) ∑_i 1{ Δprec_ind_{i,t} ≥ 0, Δprec_common_t ≥ 0, Δvar_{i,t} ≥ 0 }`,
    where `J_t` counts non-missing agents at `t`.
    
#### Illustration 2 — opposite movement of update measure and precision measure
* **Code:** `Stylized_facts/main_stylized_facts.m`. Results printed in the command window.

#### P-values underlying “significance” statements in correlation tables

* **Code:** `Stylized_facts/main_stylized_facts.m`
* **Mechanism:** `[rho, pval] = corr(x, y, 'rows','complete')`
* **Outputs:** `PVAL_*` matrices alongside the corresponding `CORR_*` matrices (e.g., `PVAL_VIX`, `PVAL_ANX`, `PVAL_CPI`, etc.).


## Options in `Stylized_facts/main_stylized_facts.m`

At the top of the file: - `REMOVE_SEASONALITY`: removes
deterministic quarterly seasonality from the 1-D information-measure
series using
`Stylized_facts/functions/removeQuarterlySeasonality.m`. -
`COVID_FREE_CORR`: for correlation tables only, truncates the
macro sample through 2020Q1 (defaults: `COVID_END_YEAR=2020`,
`COVID_END_QTR=1`).
