Neural Factorization Machines for Sparse Predictive Analytics
著者
Abstract
Many predictive tasks of web applications need to model categori-cal variables, such as user IDs and demographics like genders and occupations. To apply standard machine learning techniques, these categorical predictors are always converted to a set of binary fea-tures via one-hot encoding, making the resultant feature vector highly sparse. To learn from such sparse data eectively, it is crucial to account for the interactions between features.
Factorization Machines (FMs) are a popular solution for eciently using the second-order feature interactions. However, FM mod-els feature interactions in a linear way, which can be insucient for capturing the non-linear and complex inherent structure of real-world data. While deep neural networks have recently been applied to learn non-linear feature interactions in industry, such as the Wide&Deep by Google and DeepCross by Microso, the deep
structure meanwhile makes them dicult to train. In this paper, we propose a novel model Neural Factorization Ma-chine (NFM) for prediction under sparse seings. NFM seamlessly combines the linearity of FM in modelling second-order feature interactions and the non-linearity of neural network in modelling
higher-order feature interactions. Conceptually, NFM is more ex-pressive than FM since FM can be seen as a special case of NFM without hidden layers. Empirical results on two regression tasks show that with one hidden layer only, NFM signicantly outper-forms FM with a 7.3% relative improvement. Compared to the recent deep learning methods Wide&Deep and DeepCross, our NFM uses a shallower structure but oers beer performance, being much
easier to train and tune in practice.
メモ
FMに関して
手作業で特徴量を拡張する方法以外にMLモデルによる相互作用の学習する方法もある
特徴量を潜在空間に埋め込み、特徴量間の相互作用は埋め込み表現の内積でモデル化される
線形性故にパフォーマンスに限界があり、相互作用もペアワイズのものだけ
現実のデータは複雑で非線形
提案手法
新しいスパースデータのための予測としてNeural Factorization Machines (NFMs)と名付けるものを提案
ハイオーダーと非線形の特徴量の相互作用に対応
Bilinear Interaction (Bi-Interaction) poolingという新しい演算を導入
貢献性
Bi-Interaction poolingのニューラルネットワークへの導入、FMへの新しいニューラルネットの観点
新しい観点からNFMモデルをニューラルネットのフレームワークのもとFMを深くする形で開発
ハイオーダー、非線形の相互作用を学習
二つの現実世界のタスクに拡張した実験