Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract more metadata using exifTool #2645

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions meshroom/nodes/aliceVision/ExtractMetadata.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
__version__ = "0.1"

from meshroom.core import desc
from meshroom.core.utils import VERBOSE_LEVEL
from pathlib import Path

import pyalicevision as av

import distutils.dir_util as du
import shutil
import glob
import os
import subprocess

Check warning on line 13 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L9-L13

Added lines #L9 - L13 were not covered by tests


class ExtractMetadata(desc.Node):
size = desc.DynamicNodeSize("input")

Check warning on line 17 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L16-L17

Added lines #L16 - L17 were not covered by tests

category = 'Utils'
documentation = '''

Check warning on line 20 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L19-L20

Added lines #L19 - L20 were not covered by tests
Using exifTool, this node extracts metadata of all images referenced in a sfmData and store them in appropriate files.
'''

inputs = [

Check warning on line 24 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L24

Added line #L24 was not covered by tests
desc.File(
name="input",
label="Input",
description="SfMData file input.",
value="",
),
desc.BoolParam(
name="keepFilename",
label="Keep Filename",
description="Keep the filename of the inputs for the outputs.",
value=False,
),
desc.ChoiceParam(
name="extension",
label="Output File Extension",
description="Metadata file extension.",
value="txt",
values=["txt", "xml", "xmp"],
exclusive=True,
),
desc.StringParam(
name="arguments",
label="Arguments",
description="ExifTool command arguments",
value="",
),
desc.BoolParam(
name="insertInSfm",
label="Update sfmData",
description="Insert the extracted metadata in the sfmData file.",
value=False,
),
desc.ChoiceParam(
name="verboseLevel",
label="Verbose Level",
description="Verbosity level (fatal, error, warning, info, debug, trace).",
values=VERBOSE_LEVEL,
value="info",
),
]

outputs = [

Check warning on line 66 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L66

Added line #L66 was not covered by tests
desc.File(
name="output",
label="Result Folder",
description="Output path for the resulting metadata files.",
value=desc.Node.internalFolder,
),
]

def processChunk(self, chunk):
try:
chunk.logManager.start(chunk.node.verboseLevel.value)

Check warning on line 77 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L75-L77

Added lines #L75 - L77 were not covered by tests

if chunk.node.input.value == "" or chunk.node.input.value[-4:].lower() != '.sfm':
error = 'This node need to have a sfmData connected as input.'
chunk.logger.error(error)
raise RuntimeError(error)

Check warning on line 82 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L79-L82

Added lines #L79 - L82 were not covered by tests

if not os.path.exists(chunk.node.output.value):
os.mkdir(chunk.node.output.value)

Check warning on line 85 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L84-L85

Added lines #L84 - L85 were not covered by tests

dataAV = av.sfmData.SfMData()
if av.sfmDataIO.load(dataAV, chunk.node.input.value, av.sfmDataIO.ALL):
views = dataAV.getViews()
for id, v in views.items():
inputFile = v.getImage().getImagePath()
if chunk.node.keepFilename.value:
outputMetadataFilename = os.path.join(chunk.node.output.value, Path(inputFile).stem + "." + chunk.node.extension.value)

Check warning on line 93 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L87-L93

Added lines #L87 - L93 were not covered by tests
else:
outputMetadataFilename = os.path.join(chunk.node.output.value, str(id) + "." + chunk.node.extension.value)

Check warning on line 95 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L95

Added line #L95 was not covered by tests

if chunk.node.extension.value == 'txt':
cmd = 'exiftool ' + chunk.node.arguments.value.strip() + ' ' + inputFile + ' > ' + outputMetadataFilename
elif chunk.node.extension.value == 'xml':
cmd = 'exiftool -X ' + chunk.node.arguments.value.strip() + ' ' + inputFile + ' > ' + outputMetadataFilename

Check warning on line 100 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L97-L100

Added lines #L97 - L100 were not covered by tests
else: #xmp
cmd = 'exiftool -tagsfromfile ' + inputFile + ' ' + chunk.node.arguments.value.strip() + ' ' + outputMetadataFilename
chunk.logger.debug(cmd)
error = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read().decode()

Check failure on line 104 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

codefactor.io / CodeFactor

meshroom/nodes/aliceVision/ExtractMetadata.py#L104

subprocess call with shell=True identified, security issue. (B602)
chunk.logger.debug(error)
if error != "":
chunk.logger.error(error)
raise RuntimeError(error)
if not os.path.exists(outputMetadataFilename):
info = 'No metadata extracted for file ' + inputFile
chunk.logger.info(info)
elif chunk.node.insertInSfm.value:
cmd = 'exiftool ' + chunk.node.arguments.value.strip() + ' ' + inputFile
metadata = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.read().decode()

Check failure on line 114 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

codefactor.io / CodeFactor

meshroom/nodes/aliceVision/ExtractMetadata.py#L114

subprocess call with shell=True identified, security issue. (B602)
chunk.logger.debug(metadata)
lmeta = metadata.split('\n')
for i in range(1, len(lmeta)-1):
l = lmeta[i].split(':', 1)

Check notice on line 118 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

codefactor.io / CodeFactor

meshroom/nodes/aliceVision/ExtractMetadata.py#L118

Ambiguous variable name 'l'. (E741)
v.getImageInfo().addMetadata('ExifTool:'+l[0].strip(), l[1].strip())

Check warning on line 119 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L102-L119

Added lines #L102 - L119 were not covered by tests

if chunk.node.insertInSfm.value:
outputSfm = os.path.join(chunk.node.output.value, Path(chunk.node.input.value).stem + ".sfm")
av.sfmDataIO.save(dataAV, outputSfm, av.sfmDataIO.ALL)

Check warning on line 123 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L121-L123

Added lines #L121 - L123 were not covered by tests

chunk.logger.info('Metadata extraction end')

Check warning on line 125 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L125

Added line #L125 was not covered by tests

finally:
chunk.logManager.end()

Check warning on line 128 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L128

Added line #L128 was not covered by tests