Strategy/projects/files/splitting/article3_submission_plan.md
+

article3_submission_plan

PhD Article 3: Splitting — Submission Plan

Created: 2026-03-16 (worker)
Paper: “Stochastic gradient algorithms from ODE splitting perspective”
Authors: Daniil Merkulov, Ivan Oseledets
Base file: /root/Strategy/projects/files/splitting/split-sgd/main.tex (610 lines, ~14pp)
Discuss with Oselodets: April 2026 meeting

Current Status

The arxiv-style paper exists and covers:
- SGD as first-order splitting scheme for gradient-flow ODE
- Theorem (LLS local solution): analytical formula for linear least squares local ODE
- Global upper bound on splitting error (Lemma + Theorem)
- Applications: LLS, Logistic Regression, Softmax Regression
- Kaczmarz as limit case of splitting (unit batch size, h→∞)
- Empirical experiments on softmax regression

Added 2026-03-16 (746 lines):
- LASSO section: Theorem A (LT zero bias, sketch), Theorem D (Strang bias bound, sketch), C_D Remark (conjecture + numerics), Corollary (splitting order matters)

Added 2026-03-17 (main_jcam.tex, 20pp):
- Full proof of Theorem A (4-step: descent lemma, prox optimality, one-step bound, telescoping)
- Proof sketch of Theorem D (4-step: modified objective, ISTA bound, fixed-point shift, combining)
- C_D = ‖A·sign(θ*)‖²/8 stated in theorem (not just Remark)

Still missing for journal submission:
1. ✗ Rigorous proof of Theorem D (proof sketch present; Steps 3-4 approximations need exact tracking)
2. ✗ Extended Introduction/Related Work (positioning vs Blanes/Casas/Murua 2024)
3. ✗ Stronger numerical experiments (LASSO bias visualization, C_D estimation figure)
4. ✗ Switch from ICLR template to elsarticle

Target Journal: JCAM (Primary)

Journal: Journal of Computational and Applied Mathematics (Elsevier)
ISSN: 0377-0427
Submission: https://www.editorialmanager.com/cam/
Typical page limit: 15-25 pages, no strict limit
Style: elsarticle LaTeX class
Impact Factor: ~2.4 (Q2 Mathematics)
Review time: ~3-4 months typical

Why JCAM fits:
- Published ADI/splitting papers (Sarshar et al. 2021, JCAM)
- Special issue 2024: “Computational Methods in Deep Learning”
- Accepts: method + theory + numerical experiments format
- SGD-splitting connection under-explored in JCAM (USP)

Alternative Journals

Journal IF Fit Notes
COAP (Computational Optimization and Applications) ~2.0 High Optimization focus, Springer
OMS (Optimization Methods and Software) ~1.8 High Taylor & Francis
Numerical Algorithms (Springer) ~2.1 Medium Numerical methods
J. Scientific Computing ~2.5 Medium Broader numerical focus
Mathematics of Computation ~2.0 Lower Harder to get into

Extension Plan

1. LASSO Section (new, core contribution for journal)

Add after “Applications” section:

Section: LASSO with Strang Splitting

Problem: F(θ) = ½‖Aθ - y‖² + λ‖θ‖₁ → min

Two splitting schemes:
- LT (Lions-Teboulle / ISTA): θ_{k+1} = soft(θ_k - h·∇f(θ_k), hλ)
- Strang-ISTA: θ_{k+1} = soft(soft(θ_k, hλ/2) - h·∇f(·), hλ/2)

Key results to add:
- Theorem A: LT converges to θ with zero algorithmic bias (F(θ̄_K) - F ≤ ‖θ₀-θ‖²/(2hK))
- Theorem D: Strang-ISTA has extra floor: F(θ̄_K) - F ≤ ‖θ₀-θ‖²/(2hK) + C_D·h²·λ²
- C_D formula: Empirical C_D ≈ 0.77; analytical formula: C_D = ‖A·sign(θ)‖²/8 (VERIFY WITH OSELODETS)
- Corollary: Strang splitting introduces shrinkage bias ≈ hλ/2·sign(θ*) at fixed point

Status of proofs:
- Theorem A: sketch ✅ (needs rigorous proof)
- Theorem D: rough sketch ✅ (exact C_D derivation — discuss April)
- C_D numerical: ✅ script at scripts/article3_cd_estimation.py, C_D≈0.77

Position vs:
- Blanes/Casas/Murua (Acta Numerica 2024): 161-page review of splitting — our work is ML/SGD focused, they’re PDE focused. Note connection.
- DOSnet (JCP 2023): Deep Operator-Splitting Network — architectural use. We analyze optimization convergence, not architecture.
- Kaczmarz literature: Needell 2014, Strohmer 2009 — already cited in paper

3. Experiments Update

Current: softmax regression on MNIST
Add for journal:
- LASSO convergence comparison (LT vs Strang-ISTA) with bias visualization
- Stepsize robustness plot (Strang more robust to large h, but bias kicks in)
- C_D estimation figure (empirical vs formula)

Preparation Checklist (before April meeting)

  • LASSO section added (2026-03-16 worker): \section{LASSO with Splitting Schemes} integrated into main.tex (610 → 746 lines). Contains: problem setup, Theorem A (LT zero bias sketch), Theorem D (Strang-ISTA convergence sketch), C_D Remark (empirical 0.77 + analytic conjecture + fixed-point shift), Corollary (splitting order matters).
  • Verify C_D = ‖A·sign(θ*)‖²/8 numerically: scripts/article3_cd_estimation.py ✅ C_D=0.767
  • Rigorous proof of Theorem A (LT zero bias) (2026-03-17 worker): Full proof integrated into main_jcam.tex (4 steps: descent lemma, subgradient at prox, one-step bound, telescoping). Matches Beck & Teboulle 2009, Thm 3.1. PDF: 20pp ✅.
  • Proof sketch of Theorem D (Strang bias bound) (2026-03-17 worker): Proof sketch integrated into main_jcam.tex (4 steps: modified objective + biased fixed point, ISTA-type bound, fixed-point shift, combining). C_D formula stated in theorem statement as $\|A\operatorname{sign}(\vect\theta^*)\|^2/8$.
  • Verify C_D formula analytically with Oselodets (April meeting agenda item 1)
  • Related Work update (2026-03-17 worker): 4 paragraphs added — Continuous-time analysis (su2014, wibisono, helmke, evtushenko), Splitting in numerical analysis (blanes2024splitting — PDE vs ML distinction), Splitting for DL architectures (DOSnet — arch vs optim distinction), Kaczmarz/randomised methods. biblio.bib: +2 entries (blanes2024splitting, huang2022dosnet). PDF: 27pp ✅.
  • JCAM elsarticle template (2026-03-16 worker): main_jcam.tex ✅ — \documentclass[preprint,12pt]{elsarticle}, frontmatter (\author, \ead, \address), keyword, journal, elsarticle-num bib. 19pp PDF compiles. elsarticle.cls co-located. Figures = draft placeholders (add for final submission).
  • LASSO figures generated (2026-03-16 worker): scripts/article3_lasso_figures.py → Strategy/phd/figures/: lasso_convergence.pdf, lasso_bias_floor.pdf, lasso_fixed_point_shift.pdf
  • Support expansion figures (2026-03-18 worker): scripts/article3_support_expansion.py → Fig4a: lasso_support_expansion.pdf, Fig4b: lasso_dual_variables.pdf. Key finding: Strang activates j iff dual_j > λ/2 (threshold λ/2 vs λ for LT). C_D analytic = 0.769 explains 80% of empirical 0.959. Results: Strategy/phd/results/article3_support_expansion.json.
  • All experiment figures in graphicspath (2026-03-17 worker): Copied LLS_iter/time, LLS_tom_iter/time, Logreg_iter/time, Softmax_fashion_mnist_iter/time_err0.25, upper_bound_2, upper_bound_many from /root/phd/thesis_real/Figures/ → Strategy/phd/figures/. main_jcam.tex compiles with 0 figure errors, 26 pages ✅.

April Oselodets Meeting Agenda Items

  1. C_D formula: Is C_D = ‖A·sign(θ*)‖²/8 correct? Derivation path?
  2. Theorem A proof strategy: Which convergence analysis framework to use (Lyapunov? Descent lemma?)
  3. Target journal: JCAM or COAP? Does Oselodets have JCAM connections?
  4. Co-authorship: Paper already lists Oselodets — confirm he’s still co-author for journal version
  5. Experiments: Which additional experiments are needed?

Notes

  • JCAM: Switch LaTeX to elsarticle template (see Elsevier author guidelines)
  • arxiv preprint: currently no arxiv version posted (paper is local only)
  • Deadline considerations: no hard deadline; aim Q3 2026 submission (after PhD defense process starts)
  • Strategy/projects/files/splitting/jcam_splitting_papers.md — reference paper list
  • Strategy/projects/files/splitting/splitting_literature_review_2024_2026.md — 22-paper literature review
Choose icon