Code silently fails after running model.fit()

[alexandrebiomed]

I am building a recommender system with LightFM. When the line of code model.fit() is executed, the code is immediately stopped. Sometimes, I run the code again and then it works and the model is trained (without having changed any parameters !)

I tried to change some parameters :

• How sparse my matrix is (from 50% to 99.99%)
• The number of epochs
• The number of latent factors
• The loss that I use

I've set the random.seed(42) and the random_state=42.

My interaction list is [(user_id, item_id, score)]
I build my dataset properly, and I don't have any Nan or Inf.

But when i run the code, it sometimes work, and it sometimes don't. Without any error message! I've set the verbose=True :

• When it works i see the Epoch bar going to 100%
• When it does not work, I see the Epoch bar at 0% and the code is exited / stopped.

Do you have any idea why it is happening ?

I am on Windows and using Visual Studio Code with LightFM 1.7

Here is my code :

def matrix_factorization(self, interaction_matrix: pd.DataFrame):

       dataset = LightfmMatrixFactorizer.build_dataset(interaction_matrix=interaction_matrix)


       interaction_list = [
           (user_id, item_id, score)
           for user_id in interaction_matrix.index
           for item_id in interaction_matrix.columns
           if (score := interaction_matrix.loc[user_id, item_id]) > self.threshold
       ]


       interactions, weights = dataset.build_interactions(interaction_list)

       sparsity = 1 - (interactions.nnz / (interactions.shape[0] * interactions.shape[1]))
       print(f"Sparsity: {sparsity:.2%}")

       # Initialize LightFM model (e.g., warp method for implicit feedback)
       model = LightFM(no_components=self.n_factors, loss=self.loss, random_state=42)
       
       # Train the LightFM model
       try:
           model.fit(interactions=interactions, sample_weight=weights, epochs=self.epochs, num_threads=self.num_threads, verbose = True)
       except Exception as e:
           print(f"Error occurred: {e}")

Main :

lightfm3 = LightfmMatrixFactorizer(score_percentile=70, n_factors=5, loss='bpr', epochs=10, num_threads=1)

    im = pd.read_csv("data/interaction_matrix/global_interaction_matrix.csv", index_col=0)

    print(im.head(3))

    user_factors, item_factors, approx_matrix = lightfm3.matrix_factorization(interaction_matrix=im)