Optimal querying for communication-efficient ADMM using Gaussian process regression
In distributed optimization schemes consisting of a group of agents connected to a central coordinator, the optimization algorithm often involves the agents solving private local sub-problems and exchanging data frequently with the coordinator to solve the global distributed problem. In those cases,...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-03-01
|
| Series: | Franklin Open |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2773186324000112 |
| _version_ | 1797202937367232512 |
|---|---|
| author | Aldo Duarte Truong X. Nghiem Shuangqing Wei |
| author_facet | Aldo Duarte Truong X. Nghiem Shuangqing Wei |
| author_sort | Aldo Duarte |
| collection | DOAJ |
| description | In distributed optimization schemes consisting of a group of agents connected to a central coordinator, the optimization algorithm often involves the agents solving private local sub-problems and exchanging data frequently with the coordinator to solve the global distributed problem. In those cases, the query-response mechanism usually causes excessive communication costs to the system, necessitating communication reduction in scenarios where communication is costly. Integrating Gaussian processes (GP) as a learning component to the Alternating Direction Method of Multipliers (ADMM) has proven effective in learning each agent’s local proximal operator to reduce the required communication exchange. A key element for integrating GP into the ADMM algorithm is the querying mechanism upon which the coordinator decides when communication with an agent is required. In this paper, we formulate a general querying decision framework as an optimization problem that balances reducing the communication cost and decreasing the prediction error. Under this framework, we propose a joint query strategy that takes into account the joint statistics of the query and ADMM variables and the total communication cost of all agents in the presence of uncertainty caused by the GP regression. In addition, we derive three different decision mechanisms that simplify the general framework by making the communication decision for each agent individually. We integrate multiple measures to quantify the trade-off between the communication cost reduction and the optimization solution’s accuracy/optimality. The proposed methods can achieve significant communication reduction and good optimization solution accuracy for distributed optimization, as demonstrated by extensive simulations of a distributed sharing problem. |
| first_indexed | 2024-03-07T14:27:51Z |
| format | Article |
| id | doaj.art-36255f82c8de4af1bda5365803467dff |
| institution | Directory Open Access Journal |
| issn | 2773-1863 |
| language | English |
| last_indexed | 2024-04-24T08:11:22Z |
| publishDate | 2024-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Franklin Open |
| spelling | doaj.art-36255f82c8de4af1bda5365803467dff2024-04-17T04:50:32ZengElsevierFranklin Open2773-18632024-03-016100080Optimal querying for communication-efficient ADMM using Gaussian process regressionAldo Duarte0Truong X. Nghiem1Shuangqing Wei2Division of Electrical and Computer Engineering, School of Electrical Engineering and Commputer Science, Louisiana State University, Baton Rouge, LA 70803, United States; Corresponding author.School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, AZ 86011, United StatesDivision of Electrical and Computer Engineering, School of Electrical Engineering and Commputer Science, Louisiana State University, Baton Rouge, LA 70803, United StatesIn distributed optimization schemes consisting of a group of agents connected to a central coordinator, the optimization algorithm often involves the agents solving private local sub-problems and exchanging data frequently with the coordinator to solve the global distributed problem. In those cases, the query-response mechanism usually causes excessive communication costs to the system, necessitating communication reduction in scenarios where communication is costly. Integrating Gaussian processes (GP) as a learning component to the Alternating Direction Method of Multipliers (ADMM) has proven effective in learning each agent’s local proximal operator to reduce the required communication exchange. A key element for integrating GP into the ADMM algorithm is the querying mechanism upon which the coordinator decides when communication with an agent is required. In this paper, we formulate a general querying decision framework as an optimization problem that balances reducing the communication cost and decreasing the prediction error. Under this framework, we propose a joint query strategy that takes into account the joint statistics of the query and ADMM variables and the total communication cost of all agents in the presence of uncertainty caused by the GP regression. In addition, we derive three different decision mechanisms that simplify the general framework by making the communication decision for each agent individually. We integrate multiple measures to quantify the trade-off between the communication cost reduction and the optimization solution’s accuracy/optimality. The proposed methods can achieve significant communication reduction and good optimization solution accuracy for distributed optimization, as demonstrated by extensive simulations of a distributed sharing problem.http://www.sciencedirect.com/science/article/pii/S2773186324000112Gaussian processADMMDistributed optimizationProximal operatorCommunication reduction |
| spellingShingle | Aldo Duarte Truong X. Nghiem Shuangqing Wei Optimal querying for communication-efficient ADMM using Gaussian process regression Franklin Open Gaussian process ADMM Distributed optimization Proximal operator Communication reduction |
| title | Optimal querying for communication-efficient ADMM using Gaussian process regression |
| title_full | Optimal querying for communication-efficient ADMM using Gaussian process regression |
| title_fullStr | Optimal querying for communication-efficient ADMM using Gaussian process regression |
| title_full_unstemmed | Optimal querying for communication-efficient ADMM using Gaussian process regression |
| title_short | Optimal querying for communication-efficient ADMM using Gaussian process regression |
| title_sort | optimal querying for communication efficient admm using gaussian process regression |
| topic | Gaussian process ADMM Distributed optimization Proximal operator Communication reduction |
| url | http://www.sciencedirect.com/science/article/pii/S2773186324000112 |
| work_keys_str_mv | AT aldoduarte optimalqueryingforcommunicationefficientadmmusinggaussianprocessregression AT truongxnghiem optimalqueryingforcommunicationefficientadmmusinggaussianprocessregression AT shuangqingwei optimalqueryingforcommunicationefficientadmmusinggaussianprocessregression |