Application of deep reinforced learning for long-term dynamic composition of WEB services

H.B. Moroz

Abstract


systems. Service composition is a technology, which builds a complex system by combining existing simple services. With the development of service oriented architecture and web service technology, massive web ser vices with the same function begin to spring up. These services are maintained by different organizations and have different quality of service. Thus, how to choose the appropriate service to make the whole system to deliver the best overall quality of service has become a key problem in service composition research. Furthermore, because of the complexity and dynamics of the network environment, quality of service may change over time. Therefore, how to dynamically configure the composition system to adapt to the changing environment and ensure the quality of the composed web service is another problem. To address the above challenges, we propose a service composition approach based on quality of web service rediction and reinforcement learning. Specifi cally, we use long short-term memory neural network to predict the quality of web service hrough reinforcement learning. This approach can be well adapted to a dynamic network environment.

Prombles in programming 2025; 2: 40-53


Keywords


service-oriented computing; quality of service; web service composition; composite web service; deep reinforcement learning

References


Sangsanit, W. Kurutach, S. Phoomvuthisarn, REST web service composition: A survey of automation and techniques, in: 2018 Interna tional Conference on Information Networking, ICOIN, 2018, pp. 116–121.

Kritikos, K., Pernici, B., Plebani, P., Cappiello, C., Comuzzi, M., Benbernou, S.: I, B., Kertґesz, A., Parkin, M., Carro, M.: A survey on service quality description. ACM Comput. Surv. 46(1), 1:1–1:58 (2013).

Mistry, S., Bouguettaya, A., Dong, H., Qin, A.K.: Predicting dynamic requests behavior in long-term iaas service composition. In: IEEE International Conference on Web Services (ICWS) 2015, pp. 49–56. IEEE (2015).

Wang, S., Zhu, X., Yang, F.: Efficient QoS management for qos-aware web service com position. Int. J. Web Grid Serv. 10(1), 1–23 (2014).

Zeng, L., Benatallah, B., Ngu, A.H., Dumas, M., Kalagnanam, J., Chang, H.: Qos-aware middleware for web services composition. IEEE Trans. Softw. Eng. 30(5), 311–327 (2004).

Y. Yan, P. Poizat, L. Zhao, Repair vs. recom position for broken service compositions, in: International Conference on Service-Oriented Computing, Springer, 2010, pp. 152–166.

D. Ardagna, B. Pernici, Adaptive service com position in flexible processes, IEEE Trans. Softw. Eng. 33 (6) (2007) 369–384.

F. Li, L. Zhang, Y. Liu, Y. Laili, QoS-aware ser vice composition in cloud manufacturing: a Gale-Shapley algorithm-based approach, IEEE Trans. Syst.Man Cybern. Syst. (2018) 1–12.

Y. Yan, P. Poizat, L. Zhao, Self-adaptive ser vice composition through graphplan repair, in: Web Services (ICWS), 2010 IEEE Interna tional Conference on, IEEE, 2010, pp. 624-627.

Y. Yan, P. Poizat, L. Zhao, Repairing service compositions in a changing world, in: Software Engineering Research, Management and Ap plications 2010, Springer, 2010, pp. 17–36.

M. Alrifai, D. Skoutas, T. Risse, Selecting sky line services for QoS-based web service com position, in: Proceedings of the 19th Interna tional Conference on World Wide Web, ACM, 2010, pp. 11–20.

V.A. Permadi, B.J. Santoso, Efficient skyline based web service composition with QoS awareness and budget constraint, in: 2018 In ternational Conference on Information and Communications Technology, ICOIACT, 2018, pp.855–860.

O. Hammas, S.B. Yahia, S.B. Ahmed, Adap tive web service composition insuring global QoS optimization, in: 2015 International Sym posium on Networks, Computers and Commu nications, ISNCC, IEEE, 2015, pp. 1–6.

S.K. Gavvala, C. Jatoth, G. Gangadharan, R. Buyya, QoS-aware cloud service composition using eagle strategy, Future Gener. Comput. Syst. 90 (2019) 273–290.

H. Wang, X. Zhou, X. Zhou, W. Liu, W. Li, A. Bouguettaya, Adaptive service composition based on reinforcement learning, in: Interna tional Conference on Service-Oriented Com puting, Springer, 2010, pp. 92–107.

H. Wang, G. Huang, Q. Yu, Automatic hierar chical reinforcement learning for efficient large-scale service composition, in: 2016 IEEE International Conference on Web Services, ICWS, 2016, pp. 57–64.

H. Wang, X. Chen, Q. Wu, Q. Yu, Z. Zheng, A. Bouguettaya, Integratingon-policy rein forcement learning with multi-agent tech niques for adaptive service composition, in: Service-Oriented Computing, Springer, 2014, pp. 154–168.

Y. lei, P.S. Yu, Multi-agent reinforcement learning for service composition, in: 2016 IEEE International Conference on Services Computing, SCC, 2016, pp. 790–793.

P. Kendrick, T. Baker, Z. Maamar, A. Hussain, R. Buyya, D. Al-Jumeily, An efficient multi cloud service composition using a distributed multiagentbased, memory-driven approach, IEEE Trans. Sustain. Comput. (2019) 1–13.

W. Li, J. Cao, K. Hu, J. Xu, R. Buyya, A trust based agent learning model for service compo sition in mobile cloud computing environ ments, IEEE Access 7 (2019) 34207–34226.

D. Billsus, M.J. Pazzani, Learning collabora tive information filters, in: Icml, vol. 98, 1998, pp. 46–54.

J.L. Herlocker, J.A. Konstan, A. Borchers, J. Riedl, An algorithmic framework for perform ing collaborative filtering, in: SIGIR ’99: Pro ceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, August 15–19, 1999, Berkeley, CA, USA, 1999, pp. 230–237.

H.N. Kim, A.T. Ji, I. Ha, G.S. Jo, Collabora tive filtering based on collaborative tagging for enhancing the quality of recommendation, Electron. Commer. Res. Appl. 9 (1) (2010) 73 83.

K. Chen, H. Mao, X. Shi, Y. Xu, A. Liu, Trust aware and location-based collaborative filter ing for web service QoS prediction, in: 2017 IEEE 41st Annual Computer Software and Ap plications Conference, COMPSAC, vol. 2, 2017, pp. 143–148.

C. Wu, W. Qiu, X. Wang, Z. Zheng, X. Yang, Time-aware and sparsitytolerant QoS predic tion based on collaborative filtering, in: 2016 IEEE International Conference on Web Ser vices, ICWS, 2016, pp. 637–640.

G. Zou, M. Jiang, S. Niu, H. Wu, S. Pang, Y. Gan, QoS aware web service recommendation with reinforced collaborative filtering, in: In ternational Conference on Service-Oriented Computing, Springer, 2018, pp. 430–445.

R.N. Calheiros, E. Masoumi, R. Ranjan, R. Buyya, Workload prediction using ARIMA model and its impact on cloud applications x2019; QoS IEEE Trans. Cloud Comput. 3 (4) (2015) 449–458.

I. Sutskever, O. Vinyals, Q.V. Le, Sequence to sequence learning with neural networks, in: Advances in Neural Information Processing Systems, 2014, pp. 3104–3112.

A. Graves, A.-r. Mohamed, G. Hinton, Speech recognition with deep recurrent neural net works, in: Acoustics, Speech and Signal Pro cessing (Icassp), 2013 Ieee International Con ference on, IEEE, 2013, pp. 6645–6649.

R.-G. Cirstea, D.-V. Micu, G.-M. Muresan, C. Guo, B. Yang, Correlated time series forecast ing using multi-task deep neural networks, in: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, ACM, 2018, pp. 1527–1530.

L. Yunpeng, H. Di, B. Junpeng, Q. Yong, Multi-step ahead time series forecasting for different data patterns based on LSTM recur rent neural network, in: 2017 14th Web Infor mation Systems and Applications Conference, WISA, 2017, pp. 305–310.

J.D. Hamilton, Time Series Analysis, Prince ton University Press, Princeton, NJ, 1994.

J. Contreras, R. Espinola, F.J. Nogales, A.J. Conejo, ARIMA Models to predict next-day electricity prices, IEEE Trans. Power Syst. 18 (3) (2003) 1014–1020

R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction, Adaptive computa tion and machine learning, MIT Press, 1998.

D.E. Goldberg, J.H. Holland, Genetic algo rithms and machine learning, Mach. Learn. 3 (2) (1988) 95–99. 36. R.H. Crites, A.G. Barto, Elevator group con trol using multiple reinforcement learning agents, Mach. Learn. 33 (2–3) (1998) 235–262.

Y. LeCun, Y. Bengio, G. Hinton, Deep learn ing, Nature 521 (7553) (2015) 436–444.

A. Karpathy, J. Johnson, L. Fei-Fei, Visualiz ing and understanding recurrent networks, 2015, ArXiv preprint, arXiv:1506.02078.

C. Goller, A. Kuchler, Learning task-depend ent distributed representations by backpropa gation through structure, in: Neural Networks, 1996, IEEE International Conference on, vol. 1, IEEE, 1996, pp. 347–352.

S. Hochreiter, J. Schmidhuber, Long short term memory, Neural Comput. 9 (8) (1997) 1735–1780.

A. Plaat, Deep Reinforcement Learning, Springer Nature, 2022.


Refbacks

  • There are currently no refbacks.