A click-through rate prediction model based on graph neural networks and attention

doi:10.3969/j.issn.1000-1565.2026.01.010

Abstract

Abstract: To improve the prediction accuracy of click-through rate models by leveraging the high-order interactions among features, we propose a click-through rate(CTR)prediction model based on graph neural networks(GNN)and attention, called VBGA(vector-wise and Bit-wise interaction model based on GNN and attention). This model utilizes GNNs and attention mechanisms to learn fine-grained weights for each feature individually. These fine-grained feature weights are then input into both vector-level interaction layers and element-level interaction layers to jointly predict CTR. The VBGA model mainly consists of a vector-level interaction layer and an element-level interaction layer. The vector-level interaction layer constructs vector-level feature interactions using a directed graph, achieving non-repetitive explicit feature interactions, which not only reduces computational complexity but also enables higher-order- 引用格式:张玲,彭芯钰,章瑞环,等.基于Winkler地基梁理论的拼宽公路新老路基差异沉降分析［J］.河北大学学报(自然科学版),2026,46(1):1-12.引用格式:张峰,张涛,花强,等.基于图神经网络和注意力的点击率预测模型［J］.河北大学学报(自然科学版),2026,46(1):93-103.DOI:10.3969/j.issn.1000-1565.2026.01.010基于图神经网络和注意力的点击率预测模型张峰,张涛,花强,董春茹,朱杰(河北大学数学与信息科学学院,河北保定 071002)摘要:为了充分利用特征间的高阶交互以提升点击率预测模型的预测精度,提出了一种基于图神经网络和注意力的点击率预测模型VBGA(vector-wise and bit-wise interaction model based on GNN and attention),该模型借助图神经网络和注意力机制,为每个特征分别学习一个细粒度的权重,并将这种细粒度的特征权重输入到向量级交互层和元素级交互层联合预测点击率.VBGA 模型主要由向量级交互层和元素级交互层构成,其中向量级交互层采用有向图来构建向量级的特征交互,实现无重复的显式特征交互,在减少计算量的同时,还可以实现更高阶的特征交叉,以获得更准确的预测精度.此外,本文还提出了一种交叉网络用于构建元素级特征交互.在Criteo和Avazu数据集上,与其他几种最先进的点击率预测模型进行了比较,实验结果表明,VBGA可以获得良好的预测结果.关键词:点击率预测;注意力机制;图神经网络;多阶特征交互中图分类号:TP391 文献标志码:A 文章编号:1000-1565(2026)01-0093-11DOI:10.3969/j.issn.1000-1565.2026.01.010A click-through rate prediction model based on graph neural networks and attention ZHANG Feng, ZHANG Tao, HUA Qiang, DONG Chunru, ZHU Jie (College of Mathematics and Information Science, Hebei University, Baoding 071002, China)Abstract: To improve the prediction accuracy of click-through rate models by leveraging the high-order interactions among features, we propose a click-through rate(CTR)prediction model based on graph neural networks(GNN)and attention, called VBGA(vector-wise and Bit-wise interaction model based on GNN and attention). This model utilizes GNNs and attention mechanisms to learn fine-grained weights for each feature individually. These fine-grained feature weights are then input into both vector-level interaction layers and element-level interaction layers to jointly predict CTR. The VBGA model mainly consists of a vector-level interaction layer and an element-level interaction layer. The vector-level interaction layer constructs vector-level feature interactions using a directed graph, achieving non-repetitive explicit feature interactions, which not only reduces computational complexity but also enables higher-order- 收稿日期:2024-09-07;修回日期:2025-03-04 基金项目:科技部重点研发项目(2022YFE0196100);河北省自然科学基金项目(F2022511001);河北省高等学校科学技术研究项目(ZC2022070);河北大学高层次人才科研启动项目(521100223212) 第一作者:张峰(1976—),女,河北大学教授,博士,主要从事机器学习及应用方向研究.E-mail: fengzhang@hbu.edu.cn 通信作者:朱杰(1981—),男,河北大学副教授,博士,主要从事机器学习和计算机视觉方向研究. E-mail: zhujie@hbu.edu.cn 第1期张峰等:基于图神经网络和注意力的点击率预测模型河北大学学报(自然科学版) 第46卷feature crosses for more accurate prediction precision. Additionally, we propose a cross-network to construct element-level feature interactions. We compared VBGA with several state-of-the-art CTR prediction models on the Criteo and Avazu datasets. Experimental results demonstrate that our VBGA achieves good prediction results.

Key words: click-through rate(CTR)prediction, attention mechanism, graph neural network, multi-order feature interactions

CLC Number:

TP391

ZHANG Feng, ZHANG Tao, HUA Qiang, DONG Chunru, ZHU Jie. A click-through rate prediction model based on graph neural networks and attention[J]. Journal of Hebei University(Natural Science Edition), 2026, 46(1): 93-103.

References

[1] WANG R, SHIVANNA R, CHENG D, et al. DCNv2: Improved deep & cross network and practical lessons for web-scale learning to rank systems[C] //Proceedings of the Web Conference, 2021: 1785-1797. DOI:10.1145/3442381.3450078.
[2] RICHARDSON M, DOMINOWSKA E, RAGNO R. Predicting clicks: Estimating the click-through rate for new ads[C] //Proceedings of the 16th International Conference on World Wide Web. 2007: 521-530. DOI:10.1145/1242572.1242643.
[3] HE X, CHUA T S. Neural factorization machines for sparse predictive analytics[C] //Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017: 355-364. DOI:10.1145/3077136.3080777.
[4] QU Y, FANG B, ZHANG W, et al. Product-based neural networks for user response prediction over multi-field categorical data[J]. ACM Transactions on Information Systems, 2018, 37(1): 1-35. DOI:10.1145/3233770.
[5] CHENG H T, KOC L, HARMSEN J, et al. Wide & deep learning for recommender systems[C] //Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, 2016: 7-10. DOI:10.1145/2988450.2988454.
[6] RENDLE S. Factorization machines[C] //Proceedings of the 10th IEEE International Conference on Data Mining, 2010: 995-1000. DOI:10.1109/ICDM.2010.127.
[7] JUAN Y, ZHUANG Y, CHIN W S, LIN C J. Field-aware factorization machines for CTR prediction[C] //Proceedings of the 10th ACM Conference on Recommender Systems, 2016: 43-50. DOI:10.1145/2959100.2959134.
[8] XIAO J, YE H, HE X, et al. Attentional factorization machines: Learning the weight of feature interactions via attention networks[C] //Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017: 3119-3125. DOI:10.24963/ijcai.2017/435.
[9] LI X, CHEN S, DONG J, et al. Decision-making context interaction network for click-through rate prediction[C] //Proceedings of the AAAI Conference on Artificial Intelligence, 2023: 5195-5202. DOI:10.1609/aaai.v37i4.25649.
[10] ZHANG X, WANG Z, DU B, et al. Deep session heterogeneity-aware network for click through rate prediction[J] //IEEE Trans Knowl Data Eng, 2024:1-14. DOI: 10.1109/TKDE.2024.3421594.
[11] GUO H, TANG R, YE Y, et al. DeepFM: A factorization-machine based neural network for CTR prediction[C] //Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017: 1725-1731. DOI:10.24963/ijcai.2017/239.
[12] WANG R, FU B, FU G, et al. Deep & cross network for ad click predictions[C] //Proceedings of the ADKDD’17, 2017: 1-7. DOI:10.1145/3124749.3124754.
[13] LIAN J, ZHOU X, ZHANG F, et al. xDeepFM: Combining explicit and implicit feature interactions for recommender systems[C] //Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018: 1754-1763. DOI:10.1145/3219819.3220023.
[14] ZHOU G, ZHU X, SONG C, et al. Deep interest network for click-through rate prediction[C] //Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 1059-1068. DOI:10.1145/3219819.3219823.
[15] ZHAI P, YANG Y, ZHANG C. Causality-based CTR prediction using graph neural networks[J]. Information Processing & Management, 2023,60(1): 103137. DOI:10.1016/j.ipm.2022.103137.
[16] LI Z, CUI Z, WU S, et al. Fi-gnn: Modeling feature interactions via graph neural networks for ctr prediction[C] //Proceedings of the 28th ACM international conference on information and knowledge management, 2019: 539-548.DOI:10.1145/3357384.3357951
[17] LI Z, WU S, CUI Z, ZHANG X. GraphFM: Graph Factorization Machines for feature interaction modeling[J]. arXiv preprint, 2021. DOI:10.48550/arXiv.2105.11866.
[18] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141. DOI:10.1109/CVPR.2018.00745.
[19] YU F, LIU Z, LIU Q, et al. Deep interaction machine: A simple but effective model for high-order feature interactions[C] //Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020: 2285-2288. DOI:10.1145/3340531.3412077.
[20] BLONDEL M, FUJINO A, UEDA N, ISHIHATA M. Higher-order factorization machines[C] //Advances in Neural Information Processing Systems, 2016: 3351-3359. DOI:10.5555/3157382.3157473.
[21] PAN J, XU J, LOBOS RUIZ A, et al. Field-weighted factorization machines for click-through rate prediction in display advertising[C] //Proceedings of the 2018 World Wide Web Conference, 2018: 1349-1357. DOI:10.1145/3178876.3186040.
[22] SUN Y, PAN J, ZHANG A, et al. FM2: Field-matrixed factorization machines for recommender systems[C] //Proceedings of the Web Conference 2021, 2021: 2828-2837. DOI:10.1145/3442381.3449930.
[23] SONG W, SHI C, XIAO Z, et al. AutoInt: Automatic feature interaction learning via self attentive neural networks[C] //Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019: 1161-1170. DOI:10.1145/3357384.3357925.
[24] HUANG T, ZHANG Z, ZHANG J. FiBiNET: Combining feature importance and bilinear feature interaction for click-through rate prediction[C] //Proceedings of the 13th ACM Conference on Recommender Systems, 2019: 169-177. DOI:10.1145/3298689.3347043.
[25] TIAN Z, BAI T, ZHANG Z, et al. Directed acyclic graph factorization machines for CTR prediction via knowledge distillation[C] //Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 2023: 1-9. DOI:10.1145/3539597.3570384.
[26] KINGMA D P, BA J. Adam: A method for stochastic optimization[C] //arXiv preprint, 2014. DOI:10.48550/arXiv.1412.6980. (