系统管理学报

系统管理学报 ›› 2023, Vol. 32 ›› Issue (1): 118-129.DOI: 10.3969/j.issn.1005-2542.2023.01.011

• 数字经济与金融工程 • 上一篇    下一篇

基于多头注意力机制的BM-Linear信用贷款评估模型

赵雪峰1，吴德林1，吴伟伟2，王世璇1，龙森1   

  1. 1.哈尔滨工业大学（深圳） 经济管理学院，广东 深圳 518055；2.哈尔滨工业大学 经济与管理学院，哈尔滨 150001
  • 出版日期:2023-01-28 发布日期:2023-01-17
  • 作者简介:赵雪峰（1993-），男，博士生。研究方向为创新管理、深度学习、机器学习。
  • 基金资助:
    国家自然科学基金面上项目(72072047)；黑龙江省哲学社会科学研究规划项目(19GLB087)；教育部人文社会科学研究青年基金资助项目(20YJC630090)；中央高校基本科研业务费专项资金资助项目 (HIT.HSS.202139，HIT.HSS.202102)；广东省基础与应用基础研究基金联合基金青年基金资助项目(2019A1515110955)；深圳市基础研究专项（自然科学基金）面上项目(JCYJ20190806144607277)

A BM-Linear Credit Loan Evaluation Model Based on Multi-Head Attention

ZHAO Xuefeng1, WU Delin1, WU Weiwei2, WANG Shixuan1, LONG Sen1   

  1. 1.School of Economics and Management, Harbin Institute of Technology(Shenzhen), Shenzhen 518055, Guangdong, China;2.School of Economics and Management, Harbin Institute of Technology, Harbing 150001, China
  • Online:2023-01-28 Published:2023-01-17

PDF

摘要：

信贷评估模型可加快放贷效率、缩减放贷时间。利用Pytorch深度学习框架，组合Bag-of-Words及Bert中多头注意力机制得到BM-Linear评估模型，同时在引入多组信贷训练集的前提下，创造性地构建参数独立训练及参数共用训练的对比实验，探究BM-Linear的优异性。研究表明：BM-Linear首先弱化与信贷训练集的对应关系，解决信贷模型受限于信贷场景问题，减少因反复训练模型所造成的放贷效率低下现象；其次，忽略缺失特征并将离散特征转为信贷文本，降低特征处理造成的信贷干扰，提高信贷评估效率；最后，克服因词袋与信贷词语对应关系所带来的词向量固化问题，实现动态词向量过程，进而提高评估准确率。所提出的BM-Linear模型，可为信贷机构高效评估快速放贷提供支持。

关键词:

多头注意力机制, Bert, Bag-of-Words, 信用贷款, 深度学习

Abstract:

The credit evaluation model can accelerate the lending efficiency and reduce the lending time. This paper uses the Pytorch deep learning framework and combines bag of words and the multi-head attention mechanism in Bert to obtain the BM-Linear evaluation model. On the premise of introducing credit training set, it conducts a comparative experiment of parameter independent training and parameter sharing training. The experiment shows that BM-Linear weakens the correspondence with the credit training set, solves the problem that the credit model is limited by the credit scenario and reduces the low lending efficiency caused by the repeated training model. In addition, BM-Linear ignores missing features and converts discrete features into credit text, which reduces the credit interference caused by feature processing. Moreover, BM-Linear overcomes the problem of word vector solidification caused by the correspondence between bag-of-words and credit words, realizes the dynamic word vector process, which improves the evaluation accuracy. The BM-Linear evaluation model proposed in this paper can provide support for efficient evaluation and rapid lending of credit institutions.

Key words: multi-head attention, Bert, Bag-of-Words, credit loan, deep learning

中图分类号: