2020-09-02

Python Use List Comprehensions and matrix data structure to solve questions

Introduction

I had three files in hand, named (English subtitles.txt, Japanese subtitles, Romanji Subtitles), need to be composed as one file. And, every row in a file, say English file, need to be followed with other two files separately.

Here are the files:

小さな小さな出来事に 悩んでいたのは何故だろう
後から後から溢れてく 涙が頬を伝うよ
「夢」なんてことを大げさに 捉えすぎていたのかなぁ
君にもらった言葉たち それだけ信じてた
久々に見た故郷の空 あの頃の「夢」に ほらまた私包まれた

Why are you worried about small little events
The tears flow over my cheeks
Was it too big to capture something like a “dream”?
The words I gave to you I just believed
The sky of my hometown I saw for a long time After all the “dream” of those days I was wrapped in myself
I remember the temperature transmitted to your smile and palm
I like it so much I say it now
I keep writing and the diary is what I wanted was the wind of melody

Chīsana chīsana dekigoto ni nayande ita no wa nazedarō
Ato kara ato kara afurete ku namida ga hoho o tsutau yo
`Yume’ nante koto o ōgesa ni torae sugite ita no ka nā
Kimi ni moratta kotoba-tachi soredake shinji teta
Hisabisa ni mita furusato (furusato) no sora anogoro no `yume’ ni hora mata watashi tsutsuma reta
Kimi no egao ya tenohira ni tsutawaru ondo oboe teru
Kon’nani sukide ite kureru ima iu yo arigatō
Kaki tsudzukete ku daiarī hoshikatta no wa merodī no kaze

expected output file:

小さな小さな出来事に 悩んでいたのは何故だろう
Chīsana chīsana dekigoto ni nayande ita no wa nazedarō
Why are you worried about small little events

後から後から溢れてく 涙が頬を伝うよ
Ato kara ato kara afurete ku namida ga hoho o tsutau yo
The tears flow over my cheeks

Using Matrix instead of multiple lists and loops

The matrix to store multiple lists is a good idea. Using columns and rows to describe new list members is easy to understand and abstract. By doing this, we can figure out that our expected output list is a comb of each column of data of matrix. Then,what we need to do next is just convert member list to string.

Using list Comprehensions to simplify code

List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.

For example, assume we want to create a list of squares, like:

>>> squares = []
>>> for x in range(10):
...     squares.append(x**2)
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Note that this creates (or overwrites) a variable named x that still exists after the loop completes. We can calculate the list of squares without any side effects using:

squares = list(map(lambda x: x**2, range(10)))

or, equivalently:

squares = [x**2 for x in range(10)]

which is more concise and readable.

A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses. The result will be a new list resulting from evaluating the expression in the context of the for and if clauses which follow it. For example, this listcomp combines the elements of two lists if they are not equal:

>>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]

and it’s equivalent to:

>>> combs = []
>>> for x in [1,2,3]:
...     for y in [3,1,4]:
...         if x != y:
...             combs.append((x, y))
...
>>> combs
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]

Note how the order of the for and if statements is the same in both these snippets.

If the expression is a tuple (e.g. the (x, y) in the previous example), it must be parenthesized.

>>> vec = [-4, -2, 0, 2, 4]
>>> # create a new list with the values doubled
>>> [x*2 for x in vec]
[-8, -4, 0, 4, 8]
>>> # filter the list to exclude negative numbers
>>> [x for x in vec if x >= 0]
[0, 2, 4]
>>> # apply a function to all the elements
>>> [abs(x) for x in vec]
[4, 2, 0, 2, 4]
>>> # call a method on each element
>>> freshfruit = ['  banana', '  loganberry ', 'passion fruit  ']
>>> [weapon.strip() for weapon in freshfruit]
['banana', 'loganberry', 'passion fruit']
>>> # create a list of 2-tuples like (number, square)
>>> [(x, x**2) for x in range(6)]
[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]
>>> # the tuple must be parenthesized, otherwise an error is raised
>>> [x, x**2 for x in range(6)]
  File "<stdin>", line 1, in <module>
    [x, x**2 for x in range(6)]
               ^
SyntaxError: invalid syntax
>>> # flatten a list using a listcomp with two 'for'
>>> vec = [[1,2,3], [4,5,6], [7,8,9]]
>>> [num for elem in vec for num in elem]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
List comprehensions can contain complex expressions and nested functions:

>>>
>>> from math import pi
>>> [str(round(pi, i)) for i in range(1, 6)]
['3.1', '3.14', '3.142', '3.1416', '3.14159']

Nested List Comprehensions

The initial expression in a list comprehension can be any arbitrary expression, including another list comprehension.

Consider the following example of a 3x4 matrix implemented as a list of 3 lists of length 4:

>>> matrix = [
...     [1, 2, 3, 4],
...     [5, 6, 7, 8],
...     [9, 10, 11, 12],
... ]

The following list comprehension will transpose rows and columns:

>>> [[row[i] for row in matrix] for i in range(4)]
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
As we saw in the previous section, the nested listcomp is evaluated in the context of the for that follows it, so this example is equivalent to:

>>>
>>> transposed = []
>>> for i in range(4):
...     transposed.append([row[i] for row in matrix])
...
>>> transposed
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]

which, in turn, is the same as:

>>> transposed = []
>>> for i in range(4):
...     # the following 3 lines implement the nested listcomp
...     transposed_row = []
...     for row in matrix:
...         transposed_row.append(row[i])
...     transposed.append(transposed_row)
...
>>> transposed
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]

In the real world, you should prefer built-in functions to complex flow statements. The zip() function would do a great job for this use case:

>>> list(zip(matrix))
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]

Code

j = []
r = []
e = []


for line in open("japanese","r",encoding="utf8").readlines():
    if line != "\n":
        j.append(line)
for line in open("romanji","r",encoding="utf8").readlines():
    if line != "\n":
        r.append(line)
r = [i for i in r[1::2]]

for line in open("english","r",encoding="utf8").readlines():
    if line != "\n":
        e.append(line)

n = ['\n' for i in range(20)]

matrix = [j,r,e,n]

row_list = [''.join(str(elem) for elem in [row[column] for row in matrix]) for column in range(e.__len__())]

f = open("lyric.txt","w+",encoding="utf8")
f.writelines(row_list)

summary

The brackets might change the order of for and if statements.