How to build a supervised machine learning pipeline?

For the transactions dataset, you need a translation of the non-English (Czech language) labels in the type column into English-language labels.

Recipe builder, column type requires labels translated to English via adding a mapping action

To do this, select “Add action” and search for “Map to text” using the search bar under “All actions”. You then add the action and configure it properly.

1. Select the input column, here it is the type column.

2. Apply the following translations:

   1. PRIJEM -> credit

   2. VYDAJ -> withdrawal

   3. VYBER -> withdrawal_cash

3. Once the configuration is completed, add the action.

Recipe builder, configuring the action “Map to Text” 

You can see the action is added to the recipe and that the preview has been updated to include the action applied to the dataset as you can now see the translated labels for column type.

Values for type column have been updated as per the added action 

Once an action is added to a recipe, if you need to change it, you have the Edit and Delete options available.

Having ensured we have added the actions we want to the recipe, finish it by clicking Run recipe and providing a name for the output dataset. After the output name is confirmed, you can see the recipe and the output dataset being generated in the Prepare data stage of the app pipeline builder.

A recipe in “Running” status with the output dataset yet to be generated

You don't need to wait for the processing, you can proceed to prepare the next dataset which is loans.

Preparing the loans dataset

Create a new recipe for loans. Apply the following actions to the recipe. 

The recipe for the loans dataset

Finish the recipe by issuing Run recipe.

After both recipes finish running, you can see both have produced prepared datasets as outputs.

The recipes for preparing the transactions and loans have finished running and the prepared datasets are available

Aggregating and joining to generate the final dataset, the ML-ready dataset

Next, you want to generate additional features to predict the loan status, this process is called feature engineering.

To do so, you need to aggregate stats of transactions over the past up to and until the loan application dates.

In turn, you should also join the transactions dataset with the loans dataset and filter out transactions that happened later than the loan date, before computing the aggregation, to avoid data leakage from the future into the present.

You can do that by selecting the prepared transactions dataset and then selecting the Merge with another dataset option this time.

Select the output dataset from the “process transactions” recipe, then join it with another dataset

This process also creates a recipe, but conveniently adds the action to merge the two datasets on your behalf.

You are shown different ways to “merge with a dataset”. Joins such as left join, outer join, and inner join, lookup values from another dataset, and aggregate-lookup. Use the “join” option.

Select the dataset to join with the “prepared transactions” dataset, which is the prepared loans dataset, then click Create recipe.

 Selecting the dataset to merge with

You will be taken to the recipe builder view with the join action open, where you can modify and confirm the join parameters inferred by the Engine.

Configure the join type as a “right” join, since you are only interested in the transactions from accounts that have a loan.

From the loans dataset, you can keep all columns except for the account_id, as the transaction dataset already has an account id.

A selected merging action is added to a new editable recipe. We can customize the action further

Save changes to add the action, you can see columns added from the prepared loans dataset as well.

Result of the join action. You can see the columns added from the loans dataset

Next, you want to remove the entries where the transaction date is past the loan date, so that you don't leak data from the future into the past.

To do this, choose the “filter rows” action within the “All actions” catalog, and enter the formula date < loan_date to keep only the rows satisfying this criterion.

The (boolean) criterion we want to filter by: Use only the transaction history that happened before the loan agreement date

Then for each loan, compute the following aggregate stats:

• Number of transactions of each type

• Minimum, maximum, mean, median, and standard deviation of:

  • amounts for each transaction type

  • interval between transactions of each type

• Days since the last transaction of each type

To facilitate finding the average interval between transactions of each type, you first want to compute the lag function of the date column, partitioned by account_id and type, ordered by the date column.

This can be accomplished with the “Shift rows…” action within the "All actions" catalog.

Name the output column produced by this action as previous_date_of_same_transaction_type

Configuring the “Shift rows” action: We want to partition by both the account_id and type columns to compute the lagged date that contains the date of the previous transaction of the same type from the same customer as the current transaction

Add this action, and then compute the number of days elapsed between date and previous_date_of_same_transaction_type using the “compute time elapsed” action, naming the output as days_since_last_transaction_of_same_type.

Note that you need to specify the optional time unit” parameter as days instead of leaving it to be the default setting of seconds.

Configuring the “Compute time elapsed” action: We want to subtract the date of the previous transaction of the same type from the date of the current transaction, to get the intervals in days between successive transactions of the same type

Then, to compute multiple stats for each transaction type, make use of the “Reshape dataset into a pivot table” action, specifying account_id for the group-by column, type for the pivot column, and credit, withdrawal, and withdrawal_cash for the pivot values. Then add multiple aggregates as follows:

  Using the “Pivot” action to compute aggregate features: We pivot by the transaction’s type to get stats of the amount and interval in days columns under the specified transaction types (pivot values) separately

After this, add a few more actions.

Completing the final recipe with the required actions: Getting the features from the loans dataset back, then getting features from the clients dataset, computing recency and age features

The reasoning behind these actions

After adding the pivot action to compute aggregates by transaction type, the columns from the “loans” dataset were lost, you can get them back by joining again using the same join keys.

You then join with the “clients” data to get the demographic features date_of_birth, age and district_id.

Then, you extract more features from the datetime columns:

• Days since last transaction of each type at loan-application day: using the “Compute time elapsed” action

• Age of the person: using the “Compute time elapsed” action, followed by division by 365 to convert days to years

As a final step, you drop all columns of DateTime type (as you have already extracted features from them) and the intermediate column storing the age in days instead of years.

You can now run this recipe to produce the output.

Output dataset of the recipe joining and aggregating from three sources

You need one more important feature from the transactions dataset: The minimum balance at the time of loan application.

To get this feature, you start again from the prepared transactions dataset. You will be joining with both the prepared loans dataset and the dataset output by the latest recipe we worked on.

Here is a summary of the recipe:

Recipe to generate the minimum balance feature and the input dataset it is created from

 Recipe to generate the balance aggregation features 

You do a right join with the prepared loans dataset and filter out transactions whose dates are later than the loan application date.

Then compute the minimum of the balance column grouped by the account_id column. Then join with the features that you generated before, using the account_id column as the key.

Selecting the training dataset

You have now produced your ML-ready training dataset. To proceed, select the final dataset and set it to Use as a training dataset. Then click Save and continue to proceed to the next step, Define what to predict.

Step 2: Selecting the target column and problem type

Select the target column. Here, it is the column named status (the loan status).

The target column needs to be specified

Once the column is chosen, the Engine automatically selects Binary classification with debt as the positive label and disables Regression, as you are predicting two classes with labels debt and no problem.

You can also configure the train/test split. For this use case, leave it as the default of 80% train (and 20% test).

 For the specified target column, the prediction type and the train/test split configuration can be chosen

Select Save and continue to proceed to the next step 3 Feature selection.

Step 3: Selecting features

Here, you can either let the Engine:

1. Automatically select the best set of features such that the top most features with a total estimated importance reaches a desired percentage threshold

2. Manually select the features with the feature-importance estimates.

Selecting features: Automatic and manual options 

If confident, you have the option to manually select any or all of them. When manually selecting the features you also have the following options to utilize to make a more informed decision.

1. Use analysis column to understand the feature statistics

2. Let the Engine estimate the feature importance and use them. Here again, you can specify which columns must not be considered as candidate features.

Option to estimate feature importance and look into analysis of the feature

Feature selection, the Engine is computing feature importance estimates

Once the estimation is done, you can sort features by importance and select the topmost features that reach a certain total percentage threshold.

Features ordered by estimated importance: Available when the Engine finishes analyzing data

In both automatic and manual options, the feature types are automatically recommended by the Engine, however you have the option to change the feature types if required. In this case, all features are Numeric; with no nominal (categorical) or descriptive (text) features.

Once you are satisfied with the selected features, you can name the feature set to reuse the same set of features later if required.

Step 4: Selecting algorithms to train ML models

In the final step of the app pipeline builder, you need to specify how the Engine should train the ML models.

You can:

• Let the Engine choose the best algorithms and train them based on estimates that satisfy certain desired criteria, or:

• You can manually choose them.

For this use case, opt for the former, letting the Engine choose 3 algorithms with an estimated prediction quality of at least 60% and an estimated training time of less than 2 hours.

Letting the Engine select algorithms. You can specify selection criteria to optimize the outcome

Select Save and continue, so that the app starts processing datasets and training models.

Finishing the process with the App summary page

You will be taken to the App summary page, showing the progress status of various steps.