Skip to content

Speeding up video frames loading #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dimaxano wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Speeding up video frames loading #52

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8a0aba7
speeding up data loading
Jun 18, 2018
5990b0a
minor api changes
dimaxano Jun 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion lipnet/lipreading/videos.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from scipy import ndimage
from scipy.misc import imresize
import skvideo.io
import cv2

@rizkiarm rizkiarm Jun 26, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle cv2 import failure (when users don't have OpenCV) as we want to give options to users to opt-out from using the feature

@dimaxano dimaxano Jun 27, 2018

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've decided to import cv2 later in the code

import dlib
from lipnet.lipreading.aligns import Align

Expand Down Expand Up @@ -191,11 +192,30 @@ def get_frames_mouth(self, detector, predictor, frames):
mouth_frames.append(mouth_crop_image)
return mouth_frames

def get_video_frames(self, path):
def get_video_frames_skvideo(self, path):

@rizkiarm rizkiarm Jun 26, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't change the current method of video loading as it has been proven to be reliable and doesn't require an additional installation of OpenCV, which is sometimes problematic (not to mention that the program itself sometimes doesn't work as expected).

Instead, provide an additional argument for the user in the from_video method. Something like this: from_video(self, path, use_open_cv=False). Please keep Sk-video to be the default method.

@dimaxano dimaxano Jun 27, 2018

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

videogen = skvideo.io.vreader(path)
#print("aaaa")

@rizkiarm rizkiarm Jun 26, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please delete this line

@dimaxano dimaxano Jun 27, 2018

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

frames = np.array([frame for frame in videogen])
return frames


def get_video_frames(self, path):

@rizkiarm rizkiarm Jun 26, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename the method to signify the use of opencv

@dimaxano dimaxano Jun 27, 2018

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

reader = cv2.VideoCapture(path)

frames = []
while True:
ret, frame = reader.read()

if not ret:
break
else:
frames.append(frame[...,::-1])

return np.array(frames)




def set_data(self, frames):
data_frames = []
for frame in frames:
Expand Down