Towards a Framework to Evaluate Generative Time Series Models for Mobility Data Features
DOI:
https://doi.org/10.5753/jisa.2024.3887Keywords:
Generative adversarial networks, time series, MobilityAbstract
Understanding human mobility has implications for several areas, such as immigration, disease control, mobile networks performance, and urban planning. However, gathering and disseminating mobility data face challenges such as data collection, handling of missing information, and privacy protection. An alternative to tackle these problems consists of modeling raw data to generate synthetic data, preserving its characteristics while maintaining its privacy. Thus, we propose MobDeep, a unified framework to compare and evaluate generative models of time series based on mobility data features, which considers statistical and deep learning-based modeling. To achieve its goal, MobDeep receives as input statistical or Generative Adversarial Network-based models (GANs) and the raw mobility data, and outputs synthetic data and the metrics comparing the synthetic with the original data. In such way, MobDeep allows evaluating synthetic datasets through qualitative and quantitative metrics. As a proof-of-concept, MobDeep implements one classical statistical model (ARIMA) and three GANs models. To demonstrate MobDeep on distinct mobility scenarios, we considered an open dataset containing information about bicycle rentals in US cities and a private dataset containing information about a Brazilian metropolis's urban traffic. MobDeep allows observing how each model performs in specific scenarios, depending on the characteristics of the mobility data. Therefore, by using MobDeep researchers can evaluate their resulting models, improving the fidelity of the synthetic data regarding the original dataset.
Downloads
References
Borji, A. (2022). Pros and cons of gan evaluation measures: New developments. Computer Vision and Image Understanding, 215:103329. DOI: 10.1016/j.cviu.2021.103329.
Brock, A., Donahue, J., and Simonyan, K. (2018). Large scale gan training for high fidelity natural image synthesis. DOI: 10.48550/arXiv.1809.11096.
Brockwell, P. J. and Davis, R. A. (2009). Time series: theory and methods. Springer Science & Business Media. Book.
Brophy, E., Wang, Z., She, Q., and Ward, T. (2023). Generative adversarial networks in time series: A systematic literature review. ACM Computing Surveys, 55(10):1-31. DOI: 10.1145/3559540.
Cao, Y.-J., Jia, L.-L., Chen, Y.-X., Lin, N., Yang, C., Zhang, B., Liu, Z., Li, X.-X., and Dai, H.-H. (2018). Recent advances of generative adversarial networks in computer vision. IEEE Access, 7:14985-15006. DOI: 10.1109/ACCESS.2018.2886814.
Chollet, F. et al. (2018). Deep learning with Python, volume 361. Manning New York. Book.
Cunha, V. C., Zavala, A. Z., Magoni, D., Inácio, P. R. M., and Freire, M. M. (2022). A complete review on the application of statistical methods for evaluating internet traffic usage. IEEE Access, 10:128433-128455. DOI: 10.1109/ACCESS.2022.3227073.
Esteban, C., Hyland, S. L., and Rätsch, G. (2017). Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.02633. DOI: 10.48550/arXiv.1706.02633.
Fanaee-T, H. and Gama, J. (2014). Event labeling combining ensemble detectors and background knowledge. Progress in Artificial Intelligence, 2(2):113-127. DOI: 10.1007/s13748-013-0040-3.
Feng, J., Yang, Z., Xu, F., Yu, H., Wang, M., and Li, Y. (2020). Learning to simulate human mobility. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery `I&' Data Mining, page 3426–3433, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3394486.3412862.
García-Jara, G., Protopapas, P., and Estévez, P. A. (2022). Improving astronomical time-series classification via data augmentation with generative adversarial networks. The Astrophysical Journal, 935(1):23. DOI: 10.3847/1538-4357/ac6f5a.
Gomes, M. F., y Piontti, A. P., Rossi, L., Chao, D., Longini, I., Halloran, M. E., and Vespignani, A. (2014). Assessing the international spreading risk associated with the 2014 west african ebola outbreak. PLoS currents, 6. DOI: 10.1371/currents.outbreaks.cd818f63d40e24aef769dda7df9e0da5.
Gonzalez, M. C., Hidalgo, C. A., and Barabasi, A.-L. (2008). Understanding individual human mobility patterns. nature, 453(7196):779-782. DOI: 10.1038/nature06958.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K. Q., editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc. Book.
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., and Alahi, A. (2018). Social gan: Socially acceptable trajectories with generative adversarial networks. DOI: 10.48550/arXiv.1803.10892.
He, M., Luo, X., Wang, Z., Yang, F., Qian, H., and Hua, C. (2020). Global traffic state recovery via local observations with generative adversarial networks. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3767-3771. IEEE. DOI: 10.1109/ICASSP40776.2020.9054656.
Helbing, D., Johansson, A., and Al-Abideen, H. Z. (2007). Dynamics of crowd disasters: An empirical study. Physical review E, 75(4):046109. DOI: 10.1103/PhysRevE.75.046109.
Hillier, B., Turner, A., Yang, T., and Park, H.-T. (2009). Metric and topo-geometric properties of urban street networks: some convergences, divergences and new results. Journal of Space Syntax Studies. Available online [link].
Ho, S., Xie, M., and Goh, T. (2002). A comparative study of neural network and box-jenkins arima modeling in time series prediction. Computers & Industrial Engineering, 42(2):371-375. DOI: 10.1016/S0360-8352(02)00036-0.
Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8):1735-1780. DOI: 10.1162/neco.1997.9.8.1735.
Huang, Z. and Tatem, A. J. (2013). Global malaria connectivity through air travel. Malaria journal, 12(1):1-11. DOI: 10.1186/1475-2875-12-269.
Iglesias, G., Talavera, E., and Díaz-'Alvarez, A. (2023). A survey on gans for computer vision: Recent research, analysis and taxonomy. Computer Science Review, 48:100553. DOI: 10.1016/j.cosrev.2023.100553.
Jauhri, A., Stocks, B., Li, J. H., Yamada, K., and Shen, J. P. (2020). Generating realistic ride-hailing datasets using gans. ACM Transactions on Spatial Algorithms and Systems (TSAS), 6(3):1-14. DOI: 10.1145/3380968.
Jeon, J., Kim, J., Song, H., Cho, S., and Park, N. (2022). Gt-gan: General purpose time series synthesis with generative adversarial networks. Advances in Neural Information Processing Systems, 35:36999-37010. DOI: 10.48550/arXiv.2210.02040.
Johansson, A., Helbing, D., Al-Abideen, H. Z., and Al-Bosta, S. (2008). From crowd dynamics to crowd safety: a video-based analysis. Advances in Complex Systems, 11(04):497-527. DOI: 10.1142/S0219525908001854.
Karras, T., Aila, T., Laine, S., and Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. DOI: 10.48550/arXiv.1710.10196.
Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P. T. P. (2016). On large-batch training for deep learning: Generalization gap and sharp minima. DOI: 10.48550/arXiv.1609.04836.
Kitamura, R., Chen, C., Pendyala, R. M., and Narayanan, R. (2000). Micro-simulation of daily activity-travel patterns for travel demand forecasting. Transportation, 27(1):25-51. DOI: 10.1023/A:1005259324588.
Kraemer, M. U., Yang, C.-H., Gutierrez, B., Wu, C.-H., Klein, B., Pigott, D. M., Du Plessis, L., Faria, N. R., Li, R., Hanage, W. P., et al. (2020). The effect of human mobility and control measures on the covid-19 epidemic in china. Science, 368(6490):493-497. DOI: 10.1126/science.abb4218.
Lei, K., Qin, M., Bai, B., Zhang, G., and Yang, M. (2019). Gcn-gan: A non-linear temporal link prediction model for weighted dynamic networks. In IEEE Conference on Computer Communications, pages 388-396. IEEE. DOI: 10.1109/INFOCOM.2019.8737631.
Lenkei, Z. (2018). Crowdsourced traffic information in traffic management: Evaluation of traffic information from waze. Available online [link].
Lin, Z., Jain, A., Wang, C., Fanti, G., and Sekar, V. (2020). Using gans for sharing networked time series data: Challenges, initial promise, and open questions. In Proceedings of the ACM Internet Measurement Conference, pages 464-483. DOI: 10.1145/3419394.3423643.
Luca, M., Barlacchi, G., Lepri, B., and Pappalardo, L. (2021). A survey on deep learning for human mobility. ACM Computing Surveys (CSUR), 55(1):1-44. DOI: 10.1145/3485125.
Malandrino, F., Chiasserini, C., and Kirkpatrick, S. (2018). Cellular network traces towards 5g: Usage, analysis and generation. IEEE Transactions on Mobile Computing, 17(3):529-542. DOI: 10.1109/TMC.2017.2737011.
Mogren, O. (2016). C-rnn-gan: Continuous recurrent neural networks with adversarial training. arXiv preprint arXiv:1611.09904. DOI: 10.48550/arXiv.1611.09904.
Mota, V. F., Cunha, F. D., Macedo, D. F., Nogueira, J. M., and Loureiro, A. A. (2014). Protocols, mobility models and tools in opportunistic networks: A survey. Computer Communications, 48:5 - 19. Opportunistic networks. DOI: 10.1016/j.comcom.2014.03.019.
Navidan, H., Moshiri, P. F., Nabati, M., Shahbazian, R., Ghorashi, S. A., Shah-Mansouri, V., and Windridge, D. (2021). Generative adversarial networks (gans) in networking: A comprehensive survey & evaluation. Computer Networks, 194:108149. DOI: 10.1016/j.comnet.2021.108149.
Piorkowski, M., Sarafijanovic-Djukic, N., and Grossglauser, M. (2009). CRAWDAD dataset epfl/mobility (v. 2009-02-24). DOI: 10.15783/C7J010.
Qu, Y., Yu, S., Zhou, W., and Tian, Y. (2020). Gan-driven personalized spatial-temporal private data sharing in cyber-physical social systems. IEEE Transactions on Network Science and Engineering, 7(4):2576-2586. DOI: 10.1109/TNSE.2020.3001061.
Rao, J., Gao, S., Kang, Y., and Huang, Q. (2020). Lstm-trajgan: A deep learning approach to trajectory privacy protection. DOI: 10.48550/arXiv.2006.10521.
Ribeiro, I., Castanheira, L., Schaeffer-Filho, A., Cordeiro, W., and Mota, V. (2021). Mobility and community detection based on topics of interest. In 2021 IEEE 18th Annual Consumer Communications Networking Conference (CCNC), pages 1-6. IEEE. DOI: 10.1109/CCNC49032.2021.9369462.
Scott, J., Gass, R., Crowcroft, J., Hui, P., Diot, C., and Chaintreau, A. (2009). CRAWDAD dataset cambridge/haggle (v. 2009-05-29). DOI: 10.15783/C70011.
Silva, T. H., De Melo, P. O. V., Almeida, J. M., and Loureiro, A. A. (2014). Large-scale study of city dynamics and urban social behavior using participatory sensing. IEEE Wireless Communications, 21(1):42-51. DOI: 10.1109/MWC.2014.6757896.
Smith, T. G. et al. (2017-). pmdarima: Arima estimators for Python. Available online [link].
Solmaz, G. and Turgut, D. (2019). A survey of human mobility models. IEEE Access, 7:125711-125731. DOI: 10.1109/ACCESS.2019.2939203.
Song, C., Qu, Z., Blumm, N., and Barabási, A.-L. (2010). Limits of predictability in human mobility. Science, 327(5968):1018-1021. DOI: 10.1126/science.1177170.
Song, H. Y., Baek, M. S., and Sung, M. (2019). Generating human mobility route based on generative adversarial network. In 2019 Federated Conference on Computer Science and Information Systems, pages 91-99. IEEE. DOI: 10.15439/2019F320.
Susskind, J., Anderson, A., and Hinton, G. E. (2010). The toronto face dataset. Technical report, Technical Report UTML TR 2010-001, U. Toronto. Non Public Dataset.
Vallender, S. (1974). Calculation of the wasserstein distance between probability distributions on the line. Theory of Probability & Its Applications, 18(4):784-786. DOI: 10.1137/1118101.
Yin, D., Yang, Q., and Ma, L. (2018). Gans based density distribution privacy-preservation on mobility data. Sec. and Commun. Netw., 2018. DOI: 10.1155/2018/9203076.
Yin, Y., Lin, Z., Jin, M., Fanti, G., and Sekar, V. (2022). Practical gan-based synthetic ip header trace generation using netshare. In ACM SIGCOMM, pages 458-472. DOI: 10.1145/3544216.3544251.
Yoon, J., Jarrett, D., and van der Schaar, M. (2019). Time-series generative adversarial networks. In Wallach, H., Larochelle, H., Beygelzimer, A., dtextquotesingle Alch'e-Buc, F., Fox, E., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. Book.
Yu, H., Li, Z., Zhang, G., Liu, P., and Wang, J. (2020). Extracting and predicting taxi hotspots in spatiotemporal dimensions using conditional generative adversarial neural networks. IEEE Transactions on Vehicular Technology, 69(4):3680-3692. DOI: 10.1109/TVT.2020.2978450.
Zhang, G. P. (2003). Time series forecasting using a hybrid arima and neural network model. Neurocomputing, 50:159-175. DOI: 10.1016/S0925-2312(01)00702-0.
Zhang, H. and Lu, X. (2020). Vehicle communication network in intelligent transportation system based on internet of things. Computer Communications, 160:799-806. DOI: 10.1016/j.comcom.2020.03.041.
Zhang, H., Wu, Y., Tan, H., Dong, H., Ding, F., and Ran, B. (2022). Understanding and modeling urban mobility dynamics via disentangled representation learning. IEEE Transactions on Intelligent Transportation Systems, 23(3):2010-2020. DOI: 10.1109/TITS.2020.3030259.
Zhang, L. (2019). Stggan: Spatial-temporal graph generation. page 608–609, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3347146.3363462.
Zheng, Y., Zhang, L., Xie, X., and Ma, W.-Y. (2009). Mining interesting locations and travel sequences from gps trajectories. In Proceedings of the 18th International Conference on World Wide Web, page 791–800, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/1526709.1526816.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Internet Services and Applications
This work is licensed under a Creative Commons Attribution 4.0 International License.
