In this tutorial, we’ll describe multiple ways in Python to read a file line by line with examples such as using readlines(), context manager, while loops, etc. After this, you can adopt one of these methods in your projects that fits the best as per the conditions.
Python has made File I/O super easy for programmers. However, it is for you to decide what’s the most efficient technique for your situation. It would depend on many parameters, such as the frequency of such an operation, the size of the file, etc.
Let’s assume we have a logs.txt file that resides in the same folder along with the Python script.
Various Techniques to Read a File Line by Line in Python
We’ll now go over each method to read a file line by line.
Readlines() to read all lines together
We recommend this solution for files with a smaller size. If the file size is large, then it becomes inefficient as it loads the entire file in memory.
However, when the file is small, it is easier to load and parse the content line by line.
The readlines() returns a sequence of all lines from the file each containing a newline char except the last one.
We’ve demonstrated the use of readlines() function in the below example. Here, you can see that we are also using the Python while loop to traverse the lines.
Example
"""
Example:
Using readlines() to read all lines in a file
"""
ListOfLines = ["Python", "CSharp", "PHP", "JavaScript", "AngularJS"]
# Function to create our test file
def createFile():
wr = open("Logs.txt", "w")
for line in ListOfLines:
# write all lines
wr.write(line)
wr.write("\n")
wr.close()
# Function to demo the readlines() function
def readFile():
rd = open ("Logs.txt", "r")
# Read list of lines
out = rd.readlines()
# Close file
rd.close()
return out
# Main test
def main():
# create Logs.txt
createFile()
# read lines from Logs.txt
outList = readFile()
# Iterate over the lines
for line in outList:
print(line.strip())
# Run Test
if __name__ == "__main__":
main()The output is as follows:
Python CSharp PHP JavaScript AngularJS
On the other hand, the above solution will cause high memory usage for large files. So, you should choose a different approach.
For instance, you may like to try this one.
Check this: Loop Through Files in a Directory
Readline() to read file line by line
When the file size reaches MBs or GB, the right idea is to fetch one line at a time. Python readline() method does this job efficiently. It does not load all data in one go.
The readline() reads the text until the newline character and returns the line. It handles the EOF (end of file) by returning a blank string.
Example
"""
Example:
Using readline() to read a file line by line in Python
"""
ListOfLines = ["Tesla", "Ram", "GMC", "Chrysler", "Chevrolet"]
# Function to create our test file
def createFile():
wr = open("Logs.txt", "w")
for line in ListOfLines:
# write all lines
wr.write(line)
wr.write("\n")
wr.close()
# Function to demo the readlines() function
def readFile():
rd = open ("Logs.txt", "r")
# Read list of lines
out = [] # list to save lines
while True:
# Read next line
line = rd.readline()
# If line is blank, then you struck the EOF
if not line :
break;
out.append(line.strip())
# Close file
rd.close()
return out
# Main test
def main():
# create Logs.txt
createFile()
# read lines from Logs.txt
outList = readFile()
# Iterate over the lines
for line in outList:
print(line.strip())
# Run Test
if __name__ == "__main__":
main()After execution, the output is:
Tesla Ram GMC Chrysler Chevrolet
Reading files using Python context manager
Python provides a concept of context managers. It involves using the “with” clause with the File I/O functions. It keeps track of the open file and closes it automatically after the file operation ends.
Hence, we can confirm that you never miss closing a file handle using the context manager. It performs cleanup tasks like closing a file accordingly.
In the below example, you can see that we are using the context manager (with) along with the Python for loop to first write and then read lines.
Example
"""
Example:
Using context manager and for loop read a file line by line
"""
ListOfLines = ["NumPy", "Theano", "Keras", "PyTorch", "SciPy"]
# Function to create test log using context manager
def createFile():
with open ("testLog.txt", "w") as wr:
for line in ListOfLines:
# write all lines
wr.write(line)
wr.write("\n")
# Function to read test log using context manager
def readFile():
rd = open ("testLog.txt", "r")
# Read list of lines
out = [] # list to save lines
with open ("testLog.txt", "r") as rd:
# Read lines in loop
for line in rd:
# All lines (besides the last) will include newline, so strip it
out.append(line.strip())
return out
# Main test
def main():
# create testLog.txt
createFile()
# read lines from testLog.txt
outList = readFile()
# Iterate over the lines
for line in outList:
print(line.strip())
# Run Test
if __name__ == "__main__":
main()After running the code snippet, the output is:
NumPy Theano Keras PyTorch SciPy
To learn more about File I/O, read our tutorial on Python file handling.
