Python Lambda with Map, filter, Data Science, and Pandas

This tutorial walks you through the Python lambda operator a.k.a anonymous function. You’ll learn how to create a lambda function and use it with lists, dictionaries, and functions like map() and filter(). In addition, we’ll cover how to use lambda in data science and with Pandas data frames with fully working examples.

But, first, let’s briefly check that Python provides the following two ways to create functions.

a) Using Def keyword: Def creates a Python function object and assigns it to a name.

b) Using Lambda: Lamdda creates an inline function and returns it as a result.

A lambda function is a lightweight anonymous function. It can accept any number of arguments but can only have a single expression. Let’s learn more about the Python lambda.

What is Lambda in Python?

Lambda is an unnamed function. It provides an expression form that generates function objects.

This expression form creates a function and returns its object for calling it later. Below is a diagram describing how the lambda works with map(), filter(), and reduce().

How to Create a Lambda Function?

In order to create lambda functions, you should first know its syntax and then take our examples to learn how to use it. Also, please note that we’ll use the following terms throughout this tutorial.

a) Lambda function
b) Lambda expression

They are one and the same thing. A lambda expression is a concise way to define an anonymous function. It is defined using the lambda keyword, followed by a list of arguments and an expression. The expression is evaluated and returned when the lambda function is called.

Syntax to Use Lambda in Python

It has the following signature:

lambda arg1, arg2, ... argN: expression using arguments

The body of a lambda function is akin to what you put in a def body’s return statement. The difference here is that the result is a typed expression, instead of explicitly returning it.

Please note that a lambda function can’t include any statements. It only returns a function object which you can assign to any variable.

The lambda statement can appear in places where the def is not allowed. For example – inside a list literal or a function call’s arguments, etc.

Single or Multiline Lambda Expression

A Python lambda expression can only be an inline expression. It cannot have its own separate body.

An inline lambda expression is a single expression, followed by a colon (:), and then the expression is evaluated and returned when the lambda function is called.

For example, this is an inline lambda expression:

lambda z: z * 2

This lambda expression takes one argument, z, and returns the product of z and 2.

A Python lambda expression can only be a single line of code. If you need to define a function with multiple lines of code, you should use a regular function definition.

Hence, a multi-line lambda expression is still an inline expression, in the sense that it is defined within a single line of code. However, the body of a Python lambda expression cannot span multiple lines. It is limited to a single expression, which means that it should be written on a single line. Check the below example, this is another way to use a lambda expression:

def is_even_or_odd(z):
  return z % 2 == 0

k = lambda z: (
    is_even_or_odd(z) and "even" or "odd"
)

print(k(4))

Meanwhile, you may like to read our tutorial on multi-line strings in Python.

Python Lambda Examples

Below are some of the examples using the Python lambda expression.

Define a lambda expression inside a list :

alist = [lambda m:m**2, lambda m,n:m*n, lambda m:m**4]

print(alist[0](10), alist[1](2, 20), alist[2](3)) # Output: 100 40 81

lambda inside a dictionary :

key = 'm'

aDict = {'m': lambda x:2*x, 'n': lambda x:3*x}

print(aDict[key](9)) # Output: 18

If you want to practice with lambda more, please refer to our Lambda usage in Python tutorial. It discusses the advantages of using a lambda function and provides a few more examples.

Python Lambda with Map(), Filter(), and Reduce()

We can extend the utility of lambda functions by using them with the filter and map functions.

It is possible by passing the lambda expression as an argument to another function. We refer to these methods as higher-order functions as they accept function objects as arguments.

Python provides two built-in functions filter() and map() which can receive lambda functions as arguments.

a) Map() Function using Lambda Expression to Modify Lists

The map() function lets us call a function on a collection or group of iterables.

We can also specify a Python lambda function in the map call as the function object.

The map() function has the following signature.

# Map() function syntax

map(function_object, iterable1, iterable2,...)

Below is a simple example illustrating the use of the map() function to convert elements of lists into uppercase.

# Python lambda demo to use map() for adding elements of two lists

alist = ['learn', 'python', 'step', 'by', 'step']

output = list(map(lambda x: x.upper() , alist))

# Output: ['LEARN', 'PYTHON', 'STEP', 'BY', 'STEP']
print(output)

Let’s have another example illustrating the use of the map() function for adding elements of two lists.

# Python lambda demo to use map() for adding elements of two lists

list1 = [1, 2, 3, 4]
list2 = [100, 200, 300, 400]

output = list(map(lambda x, y: x+y , list1, list2))

# Output: [101, 202, 303, 404]
print(output)

b) Filter() Function using Lambda Expression to Filter List Elements

The filter() function returns a list of elements filtered based on the outcome of a test function.

In Python, we can replace this test function with a lambda expression working as the function object.

The filter function has the following signature.

# Filter() function syntax

filter(function_object, list)

Below is a simple example illustrating the use of the filter() function to determine vowels from the list of alphabets.

# Python lambda demo to filter out vowles from a list

alphabets = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i']
vowels = ['a', 'e', 'i', 'o', 'u']

output = list(filter(lambda x: (x in vowels) , alphabets))

# Output: ['a', 'e', 'i']
print(output)

c) Reduce() Function using Lambda to Aggregate Data from Lists

The reduce() method works by continuously applying a function on an iterable (such as a list) until there are no items left. It produces a non-iterable result, i.e., returns a single value.

This method helps in aggregating data from a list and returning the result. It can let us do a rolling calculation over successive pairs of values in a sequence.

We can also pass a Python lambda function as an argument to the reduce method.

The reduce() function has the following syntax.

# Reduce() function syntax

reduce(func_obj, iterable[, initializer])

Below is a simple example where the reduce() method is computing the sum of elements in a list.

from functools import reduce

def fn(m, n) : return m + n

print(reduce((lambda m, n: m + n), [1, 2, 3, 4]))
print(reduce(fn, [1, 2, 3, 4]))

After executing the above code, you see the following output.

10
10

It is important to realize that you can solve more complex use cases using the lambda functions. To do that, you must continue studying the different other constructs available in Python.

Also Check: Higher Order Functions in Python

Here is a detailed explanation of how to use Python lambda for data science, explained in a way that anyone can understand.

How to use Python lambda for data science

A lambda function is a small, anonymous function that is defined without using the def keyword. It is typically used for simple, one-line operations. Lambda functions are often used in conjunction with other higher-order functions, such as map(), filter(), and sort(), to perform operations on collections of data.

Why Use Lambda Functions in Data Science?

Lambda functions are a powerful tool for data scientists because they allow you to write concise and expressive code. They are also very versatile and apply to a variety of tasks, such as:

• Data cleaning and preprocessing
• Feature engineering
• Data transformation
• Model building and evaluation

Syntax of a Lambda Function

Let’s again look at the syntax of a lambda function as follows:

lambda arguments: expression

where:

• lambda is the keyword that indicates that the function is a lambda function.
• arguments is a comma-separated list of arguments that the function takes.
• expression is a single expression that is evaluated and returned by the function.

Examples of Lambda Functions in Data Science

Here are a few examples of how to use lambda functions in data science:

Data cleaning:

# Remove whitespaces from the beginning and end of each string in a list
data = ['  Hello ', ' World!', '   How ', '   are ', '    you?']
clean_data = list(map(lambda x: x.strip(), data))
print(clean_data)

Feature engineering:

# Create a new feature by calculating the absolute value of the difference between two columns in a DataFrame
import pandas as pd

data = {'x': [1, 2, 3, 4], 'y': [5, 4, 3, 2]}
df = pd.DataFrame(data)

df['diff'] = df.apply(lambda row: abs(row['x'] - row['y']), axis=1)
print(df)

Data transformation:

# Convert all values in a list to uppercase
data = ['hello', 'world', 'how', 'are', 'you']
uppercase_data = list(map(lambda x: x.upper(), data))
print(uppercase_data)

Model building and evaluation:

In model building, lambda functions let you design personalized loss functions, activation functions, and regularization penalties. For model evaluation, they enable you to craft custom scoring functions, implement unique evaluation metrics, and assist in hyperparameter tuning. Mastering lambda functions will boost your efficiency and creativity as a data scientist.

# Define a custom scoring function for a machine learning model
from sklearn.metrics import mean_squared_error

def my_scoring_function(y_true, y_pred):
    return mean_squared_error(y_true, y_pred) ** 0.5

# Use the custom scoring function to evaluate a model
from sklearn.model_selection import cross_val_score

model = ...
scores = cross_val_score(model, X, y, scoring=my_scoring_function)
print(scores)

Tips for Using Lambda Functions Effectively

• Lambda functions should be short and concise. If the function becomes too complex, it is better to define a regular function.
• Use lambda functions with higher-order functions to avoid repetitive code.
• Be careful not to overuse lambda functions, as they can make code more difficult to read.

How to use Lambda with Pandas in Python

Lambda functions are also useful with Pandas in Python to perform various operations on DataFrames and Series.

Using lambda functions in conjunction with Pandas in Python is quite common and can be very powerful for data manipulation. Pandas provide several functions that accept functions or lambdas as arguments for data transformation. Let’s look at some examples:

1. Applying Lambda with apply() method

The apply() method in Pandas is used to apply a function along the axis of a data frame.

import pandas as pd

# Initialize a simple DataFrame
datafrm = pd.DataFrame({
    'm': [7, 5, 9, 13],
    'n': [17, 23, 31, 40]
})

# Use lambda function to double the values in column 'm'
datafrm['m'] = datafrm['m'].apply(lambda x: x * 2)

print(datafrm)

In this example, the lambda function lambda x: x * 2 is calling each element in column ‘m’, doubling its value.

2. Filtering Rows with apply() and lambda

You can use apply() with a lambda function to filter rows based on a condition.

# Filter rows where column 'B' is greater than 20
df_filtered = df[df.apply(lambda row: row['B'] > 20, axis=1)]

print(df_filtered)

Here, the lambda function checks if the value in column ‘B’ is greater than 20 for each row.

3. Using map() with Lambda for Element-wise Operations

The map() function is used to substitute each value in a Series with another value.

# Use lambda function to square the values in column 'm'
datafrm['m'] = datafrm['m'].map(lambda x: x**2)

print(datafrm)

This example squares each value in column ‘m’ using the lambda function.

Also Read: How to Rename Columns in Pandas

4. Creating New Columns

You can use assign() to create new columns based on existing ones.

# Create a new column 'C' by adding columns 'A' and 'B' element-wise
datafrm = datafrm.assign(C=lambda x: x['A'] + x['B'])

print(datafrm)

Here, a new column ‘C’ is the result of a lambda function that adds the values in columns ‘A’ and ‘B’.

5. Combining with applymap()

If you want to apply a function to each element of a data frame, you can use applymap().

# Use lambda function to square all elements in the DataFrame
datafrm = datafrm.applymap(lambda x: x**2)

print(datafrm)

This squares each element in the entire data frame.

Lambda with Pandasis applicable in various scenarios to perform custom operations on your data. The key is to understand the structure of your data and apply lambda functions where they make sense for your specific use case.

By mastering the use of lambda functions with Pandas, you can enhance your data manipulation skills and improve the efficiency and clarity of your data science projects.

Conclusion

Today, we covered the Python lambda function in detail. It is one of many ways to create functions in Python. We hope you learned it well. Try to use it in your routine programming tasks.

If you want us to continue writing such tutorials, support us by sharing this post on your social media accounts like Facebook / Twitter. This will encourage us and help us reach more people.

Best,
Happy Coding