Aligning Language Models with Investor and Market Behavior for Financial Recommendations

📝 Paper Summary

Financial Recommendation Systems Personalized LLMs Knowledge Graph-enhanced LLMs Federated Learning

FLARKO integrates Large Language Models with structured Knowledge Graphs of user history and market data, optimized via KTO to generate profitable, behaviorally aligned financial recommendations.

Core Problem

Financial recommendation systems often fail because users disregard theoretically optimal advice that conflicts with their personal preferences, and institutions cannot centralize sensitive data for training due to regulations.

Why it matters:

Purely profit-maximizing advice is ineffective if investors do not follow it due to misalignment with their behavioral patterns or ethical views.
Regulatory constraints (e.g., GDPR) prevent the centralization of sensitive client data required to train powerful personalized models.
Vanilla LLMs lack transparency and grounding, leading to hallucinations or advice that ignores specific user constraints.

Concrete Example: A standard model might recommend a high-return oil stock to an ESG-focused investor. The investor ignores this 'optimal' advice because it conflicts with their values. FLARKO uses the investor's transaction history KG to recognize this preference and recommends a renewable energy stock instead, ensuring the advice is both profitable and actually adopted.

Key Novelty

Financial Language-model for Asset Recommendation with Knowledge-graph Optimization (FLARKO)

Encodes user transaction history (intent) and market data (trends) into structured Personal and Market Knowledge Graphs (PKGs/MKGs) to ground the LLM's reasoning.
Applies Kahneman-Tversky Optimization (KTO) to fine-tune the LLM, using binary labels of 'profitable AND behaviorally aligned' rather than complex ranking data.
Deploys a federated variant (FedFLARKO) that allows institutions to collaboratively train these aligned models without sharing raw user data.

Evaluation Highlights

FLARKO consistently outperforms state-of-the-art baselines on behavioral alignment (Pref@3) and joint profitability (Comb@3) on the FAR-Trans dataset.
Mid-sized models (1.7B–4B parameters) often outperform larger 8B models, demonstrating resource efficiency suitable for real-world deployment.
FedFLARKO (federated) shows robust performance under non-IID conditions, improving with larger models despite data heterogeneity.

Breakthrough Assessment

8/10

Significant for combining KGs, LLMs, and KTO in a specific high-stakes domain (finance). First use of KTO for financial asset recommendation and first federated KG-LLM approach for this task.

⚙️ Technical Details

Problem Definition

Setting: Generating financial asset recommendations that maximize both future profitability and alignment with historical user behavior.

Inputs: User transaction history (Personal Knowledge Graph), market data (Market Knowledge Graph), and user request.

Outputs: Natural language asset recommendation.

Pipeline Flow

Data Construction (PKG & MKG creation)
Prompt Construction (JSON-LD Serialization)
LLM Inference/Generation

System Modules

KG Constructor (Data Processing)

Converts raw transaction logs and price history into Personal (PKG) and Market (MKG) Knowledge Graphs.

Model or implementation: Rule-based graph construction

Serializer (Data Processing)

Serializes the constructed KGs into JSON-LD format for the LLM context window.

Model or implementation: JSON-LD Serializer

Recommender

Generates the final asset recommendation based on the graph context and user request.

Model or implementation: Qwen family LLMs (0.6B to 8B parameters)

Novel Architectural Elements

Dual-KG Context injection: Simultaneously grounding the LLM in a Personal KG (behavior) and Market KG (external data) via JSON-LD serialization within the prompt.

Modeling

Base Model: Qwen family (0.5B, 1.5B, 4B, 7B, 14B, 32B, 72B - paper mentions specifically 0.6B to 8B in evaluation highlights, likely referring to Qwen1.5 or Qwen2 sizes closer to 0.5/1.8/4/7B)

Training Method: Kahneman-Tversky Optimization (KTO)

Objective Functions:

Purpose: Align model outputs with human utility using binary labels.

Formally: KTO loss function (implicitly referenced as maximizing utility based on prospect theory).

Adaptation: Parameter-efficient fine-tuning (implied for federated setting efficiency, though specific method like LoRA not explicitly detailed in extract)

Training Data:

FAR-Trans dataset
Binary desirability labels: 'True' if recommendation is profitable AND consistent with user behavior, 'False' otherwise.

Compute: Not reported in the paper

Comparison to Prior Work

vs. FinGPT: FLARKO uses structured KGs for specific user personalization rather than just processing unstructured text/news.
vs. FinDKG: FLARKO focuses on individual personalized asset recommendation (micro) rather than global economic forecasting (macro).
vs. Standard Recommenders: FLARKO integrates LLM reasoning with KGs to capture non-linear, evolving preferences and supports natural language explanation.
+ 1 more
vs. Direct Preference Optimization (DPO) [not cited in paper]: FLARKO uses KTO which requires only binary labels (easier to collect in finance) rather than paired preference data.

Limitations

Reliance on the accuracy of the underlying KGs; if KGs are incomplete, recommendations suffer.
Token context limits restrict the size of the historical window (capped at 5,000 triples).
Requires binary labels for KTO which implies a strict definition of 'desirable' (profitable + aligned).
Evaluation is limited to the FAR-Trans dataset.

Reproducibility

Code availability is not provided in the text. The FAR-Trans dataset is cited as an existing resource. Specific hyperparameters (learning rate, batch size) are not detailed in the provided text.

📊 Experiments & Results

Evaluation Setup

Asset recommendation using historical transaction and price data.

Benchmarks:

FAR-Trans dataset (Financial Asset Recommendation)

Metrics:

Pref@3 (Behavioral Alignment)
Comb@3 (Joint Profitability and Alignment)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
FLARKO demonstrates superior alignment and joint performance compared to baselines.
FAR-Trans	Pref@3	Not reported in the paper	Not reported in the paper	Not reported in the paper
FAR-Trans	Performance (General)	Not reported in the paper	Not reported in the paper	Not reported in the paper

Experiment Figures

Comparison of FLARKO against state-of-the-art baselines on FAR-Trans dataset.

Main Takeaways

FLARKO consistently outperforms baselines on behavioral alignment (Pref@3) and joint profitability (Comb@3), validating the dual-objective approach.
Performance does not scale linearly with model size; mid-sized models (1.7B-4B) are optimal, suggesting efficiency for real-world deployment.
FedFLARKO remains robust in federated settings with non-IID data, enabling cross-institutional collaboration without data sharing.
Using KTO with binary labels (profitable & aligned) is effective and more practical than ranking-based alignment for this domain.

📚 Prerequisite Knowledge

Prerequisites

Knowledge Graphs (structure and serialization)
Large Language Models (prompting and fine-tuning)
Federated Learning concepts (Client/Server, non-IID data)
Financial metrics (profitability, behavioral alignment)

Key Terms

KTO: Kahneman-Tversky Optimization—an alignment method for LLMs that uses binary (good/bad) labels rather than paired preferences, based on prospect theory.

PKG: Personal Knowledge Graph—a graph representation of a specific user's past transactions (buys/sells), serving as a proxy for their preferences and intent.

MKG: Market Knowledge Graph—a graph representation of external financial signals, such as asset price trends and sector metadata.

JSON-LD: JSON for Linked Data—a format used to serialize Knowledge Graphs into text that LLMs can process while retaining semantic structure.

Pref@k: A metric measuring how many of the top-k recommended assets match the user's past behavioral patterns (alignment).

Comb@k: A joint metric combining profitability and behavioral alignment, assessing if recommendations are both money-making and consistent with user history.

non-IID: Non-Independent and Identically Distributed—data distribution where clients (e.g., banks) have different types of users/assets, making federated learning harder.

Federated Learning: A machine learning approach where models are trained across multiple decentralized edge devices or servers holding local data samples, without exchanging them.

ISIN: International Securities Identification Number—a unique code that identifies a specific financial security.