Training Neural Networks by Time-Fractional Gradient Descent

Training Neural Networks by Time-Fractional Gradient Descent

Motivated by the weighted averaging method for training neural networks, we study the time-fractional gradient descent (TFGD) method based on the time-fractional gradient flow and explore the influence of memory dependence on neural network training. The TFGD algorithm in this paper is studied via t...

Full description

Bibliographic Details
Main Authors:	Jingyi Xie, Sirui Li
Format:	Article
Language:	English
Published:	MDPI AG 2022-09-01
Series:	Axioms
Subjects:	time-fractional gradient descent training neural networks weighted averaging memory dependence
Online Access:	https://www.mdpi.com/2075-1680/11/10/507

Similar Items

Function approximation method based on weights gradient descent in reinforcement learning
by: Xiaoyan QIN, et al.
Published: (2023-08-01)

Function approximation method based on weights gradient descent in reinforcement learning
by: Xiaoyan QIN, Yuhan LIU, Yunlong XU, Bin LI
Published: (2023-08-01)

Forecasting Economic Growth of the Group of Seven via Fractional-Order Gradient Descent Approach
by: Xiaoling Wang, et al.
Published: (2021-10-01)

Damped Newton Stochastic Gradient Descent Method for Neural Networks Training
by: Jingcheng Zhou, et al.
Published: (2021-06-01)

The Improved Stochastic Fractional Order Gradient Descent Algorithm
by: Yang Yang, et al.
Published: (2023-08-01)

Fractional order system identification using a joint multi-innovation fractional gradient descent algorithm
by: Zishuo Wang, et al.
Published: (2024-12-01)

Mutual Information Based Learning Rate Decay for Stochastic Gradient Descent Training of Deep Neural Networks
by: Shrihari Vasudevan
Published: (2020-05-01)

Deep ocular tumor classification model using cuckoo search algorithm and Caputo fractional gradient descent
by: Abduljlil Abduljlil Ali Abduljlil Habeb, et al.
Published: (2024-03-01)

Fractional Derivative Gradient-Based Optimizers for Neural Networks and Human Activity Recognition
by: Oscar Herrera-Alcántara
Published: (2022-09-01)

Forest fire risk assessment model optimized by stochastic average gradient descent
by: Zexin Fu, et al.
Published: (2025-01-01)

Fractional Gradient Optimizers for PyTorch: Enhancing GAN and BERT
by: Oscar Herrera-Alcántara, et al.
Published: (2023-06-01)

Stochastic gradient descent with random label noises: doubly stochastic models and inference stabilizer
by: Haoyi Xiong, et al.
Published: (2024-01-01)

Assessment of Gradient Descent Trained Rule-Fact Network Expert System Multi-Path Training Technique Performance
by: Jeremy Straub
Published: (2021-08-01)

Variable-Parameter Impedance Control of Manipulator Based on RBFNN and Gradient Descent
by: Linshen Li, et al.
Published: (2024-12-01)

Recent Advances in Stochastic Gradient Descent in Deep Learning
by: Yingjie Tian, et al.
Published: (2023-01-01)

Additional fractional gradient descent identification algorithm based on multi-innovation principle for autoregressive exogenous models
by: Zishuo Wang, et al.
Published: (2024-08-01)

Performance Evaluation of Gradient Descent Optimizers in Estuarine Turbidity Estimation with Multilayer Perceptron and Sentinel-2 Imagery
by: Naledzani Ndou, et al.
Published: (2024-10-01)

Multi-Depth Computer-Generated Hologram Based on Stochastic Gradient Descent Algorithm with Weighted Complex Loss Function and Masked Diffraction
by: Jiale Quan, et al.
Published: (2023-03-01)

A new approach to training neural networks using natural gradient descent with momentum based on Dirichlet distributions
by: R.I. Abdulkadirov, et al.
Published: (2023-02-01)

Gradient-Descent-like Ghost Imaging
by: Wen-Kai Yu, et al.
Published: (2021-11-01)

Improved Gradient Descent Iterations for Solving Systems of Nonlinear Equations
by: Predrag S. Stanimirović, et al.
Published: (2023-01-01)

Estimation of simultaneous equation models by backpropagation method using stochastic gradient descent
by: Belén Pérez-Sánchez, et al.
Published: (2024-10-01)

Smoothing gradient descent algorithm for the composite sparse optimization
by: Wei Yang, et al.
Published: (2024-11-01)

Distributed Stochastic Gradient Descent With Compressed and Skipped Communication
by: Tran Thi Phuong, et al.
Published: (2023-01-01)

Fractional Stochastic Search Algorithms: Modelling Complex Systems via AI
by: Bodo Herzog
Published: (2023-04-01)

Modified‎ ‎Step‎ ‎Size‎ ‎for‎ ‎Enhanced‎ ‎Stochastic Gradient Descent‎: ‎Convergence and Experiments
by: Mahsa Soheil shamaee, et al.
Published: (2024-09-01)

An Effective Partitional Crisp Clustering Method Using Gradient Descent Approach
by: Soroosh Shalileh
Published: (2023-06-01)

Accelerated Gradient Descent Driven by Lévy Perturbations
by: Yuquan Chen, et al.
Published: (2024-03-01)

Pipelined Stochastic Gradient Descent with Taylor Expansion
by: Bongwon Jang, et al.
Published: (2023-10-01)

Dynamics Power Quality Cost Assessment Based on a Gradient Descent Method
by: Jingyi Zhang, et al.
Published: (2024-04-01)

Adaptive Gradient Estimation Stochastic Parallel Gradient Descent Algorithm for Laser Beam Cleanup
by: Shiqing Ma, et al.
Published: (2021-05-01)

Adaptive Stochastic Conjugate Gradient Optimization for Backpropagation Neural Networks
by: Ibrahim Abaker Targio Hashem, et al.
Published: (2024-01-01)

A Novel Sine Step Size for Warm-Restart Stochastic Gradient Descent
by: Mahsa Soheil Shamaee, et al.
Published: (2024-12-01)

Tracking of a Fixed-Shape Moving Object Based on the Gradient Descent Method
by: Haris Masood, et al.
Published: (2022-01-01)

aSGD: Stochastic Gradient Descent with Adaptive Batch Size for Every Parameter
by: Haoze Shi, et al.
Published: (2022-03-01)

Self-learning system and methods of selection of weight coefficients of neural network
by: Khalilov Durbek, et al.
Published: (2024-01-01)

Accelerated Gradient Descent Algorithm for Systems With Missing Input Data Using Inverse Auxiliary Model
by: Yongqiang Zhu
Published: (2024-01-01)

Gradient Descent Batch Clustering for Image Classification
by: Jae-Sam Park
Published: (2023-07-01)

Stochastic Gradient Descent for Kernel-Based Maximum Correntropy Criterion
by: Tiankai Li, et al.
Published: (2024-12-01)

Initial Work on the Development of a Hardware-Based Gradient Descent Trained Expert System
by: Fateen Ahmed, et al.
Published: (2022-09-01)