The following graph shows such a lstm memory block.
The blue arrows are the peephole connections. So the gates “see” the cell state(s) even if the output gate is closed.
In the following a memory block has only one memory cell. So all cell (and gate) states of the complete hidden layer can be written as a vector c(t).
Then the forward pass formulars for LSTM are (t is now an index as usual):