Abstract:

The credit evaluation model can accelerate the lending efficiency and reduce the lending time. This paper uses the Pytorch deep learning framework and combines bag of words and the multi-head attention mechanism in Bert to obtain the BM-Linear evaluation model. On the premise of introducing credit training set, it conducts a comparative experiment of parameter independent training and parameter sharing training. The experiment shows that BM-Linear weakens the correspondence with the credit training set, solves the problem that the credit model is limited by the credit scenario and reduces the low lending efficiency caused by the repeated training model. In addition, BM-Linear ignores missing features and converts discrete features into credit text, which reduces the credit interference caused by feature processing. Moreover, BM-Linear overcomes the problem of word vector solidification caused by the correspondence between bag-of-words and credit words, realizes the dynamic word vector process, which improves the evaluation accuracy. The BM-Linear evaluation model proposed in this paper can provide support for efficient evaluation and rapid lending of credit institutions.

Key words: multi-head attention, Bert, Bag-of-Words, credit loan, deep learning

摘要：

信贷评估模型可加快放贷效率、缩减放贷时间。利用Pytorch深度学习框架，组合Bag-of-Words及Bert中多头注意力机制得到BM-Linear评估模型，同时在引入多组信贷训练集的前提下，创造性地构建参数独立训练及参数共用训练的对比实验，探究BM-Linear的优异性。研究表明：BM-Linear首先弱化与信贷训练集的对应关系，解决信贷模型受限于信贷场景问题，减少因反复训练模型所造成的放贷效率低下现象；其次，忽略缺失特征并将离散特征转为信贷文本，降低特征处理造成的信贷干扰，提高信贷评估效率；最后，克服因词袋与信贷词语对应关系所带来的词向量固化问题，实现动态词向量过程，进而提高评估准确率。所提出的BM-Linear模型，可为信贷机构高效评估快速放贷提供支持。

关键词:

多头注意力机制, Bert, Bag-of-Words, 信用贷款, 深度学习