This Google Colab Notebook allows you to run the code in https://github.com/alex04072000/SingleHDR which is an implementation of the CVPR 2020 paper "Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline", which can be downloaded from https://arxiv.org/abs/2004.01179

Ashutosh Sanzgiri sanzgiri@gmail.com

!git clone https://github.com/alex04072000/SingleHDR

# The versions of packages needed was identified by crawling through github issues
!pip install tensorflow==1.10.0 
!pip install tensorlayer==1.10.1

# After installation restart runtime and continue with the next cell.

# Confirm you have the correct versions
!pip list | grep tensor

tensorboard                   1.10.0         
tensorboard-plugin-wit        1.7.0          
tensorboardcolab              0.0.22         
tensorflow                    1.10.0         
tensorflow-addons             0.8.3          
tensorflow-datasets           2.1.0          
tensorflow-estimator          2.3.0          
tensorflow-gcs-config         2.3.0          
tensorflow-hub                0.9.0          
tensorflow-metadata           0.24.0         
tensorflow-privacy            0.2.2          
tensorflow-probability        0.11.0         
tensorlayer                   1.10.1

# This is needed to install the pre-trained models to local
!pip install gdown

# This is one way to convert HDR files to LDR
# http://pfstools.sourceforge.net/man_pages.html
# http://pfstools.sourceforge.net/man_pages.html
#!sudo apt-get install pfstools
!sudo apt-get install --reinstall pfstools

# Another option to convert HDR to LDR is ImageMagick
!sudo apt-get install imagemagick

# Autotone for tonemapping with ImageMagick
!curl -o autotone "http://www.fmwconcepts.com/imagemagick/downloadcounter.php?scriptname=autotone&dirname=autotone"
!chmod +x autotone

# You can also use OpenCV
!pip install opencv-python-headless

# And this function
# Ref: https://www.learnopencv.com/high-dynamic-range-hdr-imaging-using-opencv-cpp-python/
# https://docs.opencv.org/master/d6/df5/group__photo__hdr.html#gabcbd653140b93a1fa87ccce94548cd0d
# https://docs.opencv.org/4.0.0/d2/df0/tutorial_py_hdr.html
# https://www.toptal.com/opencv/python-image-processing-in-computational-photography

import cv2
import numpy as np
from cv2 import createTonemapDrago, createTonemapMantiuk, createTonemapReinhard, createTonemap

def convert_hdr(hdrfile, ldrfile, tm_choice="D", val=2.2):

  im = cv2.imread(hdrfile, cv2.IMREAD_ANYDEPTH)
  
  if tm_choice == "D":
    tonemap = createTonemapDrago(gamma=val, bias=0.85)
  elif tm_choice == "M":
    tonemap = createTonemapMantiuk(gamma=val, scale=0.75)
  elif tm_choice == "R":
    tonemap = createTonemapReinhard(gamma=val)
  else:
    tonemap = createTonemap(gamma=val)

  ldr = tonemap.process(im)
  im2_8bit = np.clip(ldr * 255, 0, 255).astype('uint8')
  cv2.imwrite(ldrfile, im2_8bit)

# This is for plotting images side by side
!pip install ipyplot

cd SingleHDR

/content/SingleHDR/SingleHDR

# Download models and unzip
!gdown https://drive.google.com/uc?id=1e9vP8YPEjGcvXCa0Bfqwxw7qks7dH-VE
!unzip -o ckpt.zip

Run test example using model trained on synthetic data

# Run test example (model trained only on synthetic data)
# Ignore the tmalloc errors
# Note output file has .hdr extension, some image viewers like Mac Preview can open it. You can convert it to .png as shown below 

!CUDA_VISIBLEDEVICES=0 python3 test_real.py --ckpt_path_deq ckpt_deq/model.ckpt \
                                            --ckpt_path_lin ckpt_lin/model.ckpt \
                                            --ckpt_path_hal ckpt_hal/model.ckpt \
                                            --test_imgs ./imgs \
                                            --output_path output_hdrs

# Original image
!ls -l imgs

total 2892
-rw-r--r-- 1 root root 2961073 Sep 25 22:35 00000.png

# HDR version
!ls -l output_hdrs

total 5544
-rw-r--r-- 1 root root 5676415 Sep 25 22:38 00000.hdr

Explore various ways to convert HDR to LDR

# Convert HDR to PNG using pfstools
!pfsin output_hdrs/00000.hdr | pfstmo_drago03 | pfsout output_hdrs/00000_hdr_pfs.png

# Convert HDR image to PNG using imagemagick
!convert output_hdrs/00000.hdr output_hdrs/00000_hdr_im.png
# Autotone it!
!./autotone output_hdrs/00000.hdr output_hdrs/00000_hdr_im.png

# Convert HDR image to PNG using openCV
convert_hdr('output_hdrs/00000.hdr', 'output_hdrs/00000_hdr_cv_d.png', 'D', 2.2)
convert_hdr('output_hdrs/00000.hdr', 'output_hdrs/00000_hdr_cv_m.png', 'M', 2.2)
convert_hdr('output_hdrs/00000.hdr', 'output_hdrs/00000_hdr_cv_r.png', 'R', 2.2)
convert_hdr('output_hdrs/00000.hdr', 'output_hdrs/00000_hdr_cv_u.png', 'U', 2.2)

Comparisons

PFSTools

# Compare original with HDR version
import ipyplot
ipyplot.plot_images(['imgs/00000.png', 'output_hdrs/00000_hdr_pfs.png'], max_images=2, img_width=600, force_b64=True)

ImageMagick with Autotone is the best choice!

ipyplot.plot_images(['imgs/00000.png', 'output_hdrs/00000_hdr_im.png'], max_images=2, img_width=600, force_b64=True)

OpenCV options

ipyplot.plot_images(['imgs/00000.png', 'output_hdrs/00000_hdr_cv_d.png'], max_images=2, img_width=600, force_b64=True)

ipyplot.plot_images(['imgs/00000.png', 'output_hdrs/00000_hdr_cv_m.png'], max_images=2, img_width=600, force_b64=True)

ipyplot.plot_images(['imgs/00000.png', 'output_hdrs/00000_hdr_cv_r.png'], max_images=2, img_width=600, force_b64=True)

ipyplot.plot_images(['imgs/00000.png', 'output_hdrs/00000_hdr_cv_u.png'], max_images=2, img_width=600, force_b64=True)

Run test example on model trained with Real + Synthetic data

# Run test example (model trained only on real + synthetic data)
# Ignore the tmalloc errors
# Note output file has .hdr extension, but can be opened by image viewer such as preview

# this is better quality

!CUDA_VISIBLEDEVICES=0 python3 test_real_refinement.py --ckpt_path ckpt_deq_lin_hal_ref/model.ckpt \
                                                       --test_imgs ./imgs \
                                                       --output_path output_hdrs2

!pfsin output_hdrs2/00000.hdr | pfstmo_drago03| pfsout output_hdrs2/00000_hdr.png
ipyplot.plot_images(['imgs/00000.png', 'output_hdrs2/00000_hdr.png'], max_images=2, img_width=600, force_b64=True)

Upload your own file now

!mkdir -p uploads
!mv /landscape.png uploads

!CUDA_VISIBLEDEVICES=0 python3 test_real_refinement.py --ckpt_path ckpt_deq_lin_hal_ref/model.ckpt \
                                                       --test_imgs ./uploads \
                                                       --output_path output_hdrs

# Convert to HDR & Autotone it!
!./autotone output_hdrs/landscape.hdr output_hdrs/landscape_hdr_im.png
ipyplot.plot_images(['uploads/landscape.png', 'output_hdrs/landscape_hdr_im.png'], max_images=2, img_width=600, force_b64=True)

# Convert HDR to PNG using pfstools
!pfsin output_hdrs/landscape.hdr | pfstmo_drago03 | pfsout output_hdrs/landscape_hdr_pfs.png
ipyplot.plot_images(['uploads/landscape.png', 'output_hdrs/landscape_hdr_pfs.png'], max_images=2, img_width=600, force_b64=True)

# Convert HDR image to PNG using opencv
origfile = 'uploads/landscape.png'
hdrfile = 'output_hdrs/landscape.hdr'
for tonemap in ['D', 'M', 'R', 'U']:
  outfile = f'output_hdrs/landscape_hdr_cv_{tonemap}.png'
  convert_hdr(hdrfile, outfile, 'D', 2.2)
  ipyplot.plot_images([origfile, outfile], max_images=2, img_width=600, force_b64=True)

About

This notebook is a demonstration of some of capabilities of fastpages with notebooks.

With fastpages you can save your jupyter notebooks into the _notebooks folder at the root of your repository, and they will be automatically be converted to Jekyll compliant blog posts!

Front Matter

The first cell in your Jupyter Notebook or markdown blog post contains front matter. Front matter is metadata that can turn on/off options in your Notebook. It is formatted like this:

# "My Title"
> "Awesome summary"

- toc:true- branch: master- badges: true- comments: true
- author: Hamel Husain & Jeremy Howard
- categories: [fastpages, jupyter]

Setting toc: true will automatically generate a table of contents
Setting badges: true will automatically include GitHub and Google Colab links to your notebook.
Setting comments: true will enable commenting on your blog post, powered by utterances.

The title and description need to be enclosed in double quotes only if they include special characters such as a colon. More details and options for front matter can be viewed on the front matter section of the README.

Markdown Shortcuts

A #hide comment at the top of any code cell will hide both the input and output of that cell in your blog post.

A #hide_input comment at the top of any code cell will only hide the input of that cell.

The comment #hide_input was used to hide the code that produced this.

put a #collapse-hide flag at the top of any cell if you want to hide that cell by default, but give the reader the option to show it:

#collapse-hide
import pandas as pd
import altair as alt

put a #collapse-show flag at the top of any cell if you want to show that cell by default, but give the reader the option to hide it:

#collapse-show
cars = 'https://vega.github.io/vega-datasets/data/cars.json'
movies = 'https://vega.github.io/vega-datasets/data/movies.json'
sp500 = 'https://vega.github.io/vega-datasets/data/sp500.csv'
stocks = 'https://vega.github.io/vega-datasets/data/stocks.csv'
flights = 'https://vega.github.io/vega-datasets/data/flights-5k.json'

Interactive Charts With Altair

Charts made with Altair remain interactive. Example charts taken from this repo, specifically this notebook.

Example 1: DropDown

# single-value selection over [Major_Genre, MPAA_Rating] pairs
# use specific hard-wired values as the initial selected values
selection = alt.selection_single(
    name='Select',
    fields=['Major_Genre', 'MPAA_Rating'],
    init={'Major_Genre': 'Drama', 'MPAA_Rating': 'R'},
    bind={'Major_Genre': alt.binding_select(options=genres), 'MPAA_Rating': alt.binding_radio(options=mpaa)}
)
  
# scatter plot, modify opacity based on selection
alt.Chart(df).mark_circle().add_selection(
    selection
).encode(
    x='Rotten_Tomatoes_Rating:Q',
    y='IMDB_Rating:Q',
    tooltip='Title:N',
    opacity=alt.condition(selection, alt.value(0.75), alt.value(0.05))
)

Example 2: Tooltips

alt.Chart(df).mark_circle().add_selection(
    alt.selection_interval(bind='scales', encodings=['x'])
).encode(
    alt.X('Rotten_Tomatoes_Rating', type='quantitative'),
    alt.Y('IMDB_Rating', type='quantitative', axis=alt.Axis(minExtent=30)),
#     y=alt.Y('IMDB_Rating:Q', ), # use min extent to stabilize axis title placement
    tooltip=['Title:N', 'Release_Date:N', 'IMDB_Rating:Q', 'Rotten_Tomatoes_Rating:Q']
).properties(
    width=500,
    height=400
)

Example 3: More Tooltips

# select a point for which to provide details-on-demand
label = alt.selection_single(
    encodings=['x'], # limit selection to x-axis value
    on='mouseover',  # select on mouseover events
    nearest=True,    # select data point nearest the cursor
    empty='none'     # empty selection includes no data points
)

# define our base line chart of stock prices
base = alt.Chart().mark_line().encode(
    alt.X('date:T'),
    alt.Y('price:Q', scale=alt.Scale(type='log')),
    alt.Color('symbol:N')
)

alt.layer(
    base, # base line chart
    
    # add a rule mark to serve as a guide line
    alt.Chart().mark_rule(color='#aaa').encode(
        x='date:T'
    ).transform_filter(label),
    
    # add circle marks for selected time points, hide unselected points
    base.mark_circle().encode(
        opacity=alt.condition(label, alt.value(1), alt.value(0))
    ).add_selection(label),

    # add white stroked text to provide a legible background for labels
    base.mark_text(align='left', dx=5, dy=-5, stroke='white', strokeWidth=2).encode(
        text='price:Q'
    ).transform_filter(label),

    # add text labels for stock prices
    base.mark_text(align='left', dx=5, dy=-5).encode(
        text='price:Q'
    ).transform_filter(label),
    
    data=stocks
).properties(
    width=500,
    height=400
)

Data Tables

You can display tables per the usual way in your blog:

# display table with pandas
df[['Title', 'Worldwide_Gross', 
    'Production_Budget', 'Distributor', 'MPAA_Rating', 'IMDB_Rating', 'Rotten_Tomatoes_Rating']].head()

Images

Local Images

You can reference local images and they will be copied and rendered on your blog automatically. You can include these with the following markdown syntax:

![](my_icons/fastai_logo.png)

Remote Images

Remote images can be included with the following markdown syntax:

![](https://image.flaticon.com/icons/svg/36/36686.svg)

Animated Gifs

Animated Gifs work, too!

![](https://upload.wikimedia.org/wikipedia/commons/7/71/ChessPawnSpecialMoves.gif)

Captions

You can include captions with markdown images like this:

![](https://www.fast.ai/images/fastai_paper/show_batch.png "Credit: https://www.fast.ai/2020/02/13/fastai-A-Layered-API-for-Deep-Learning/")

Other Elements

GitHub Flavored Emojis

Typing I give this post two :+1:! will render this:

I give this post two :+1:!

Tweetcards

Typing > twitter: https://twitter.com/jakevdp/status/1204765621767901185?s=20 will render this:

Altair 4.0 is released! https://t.co/PCyrIOTcvv
Try it with:

pip install -U altair

The full list of changes is at https://t.co/roXmzcsT58 ...read on for some highlights. pic.twitter.com/vWJ0ZveKbZ
— Jake VanderPlas (@jakevdp) December 11, 2019

Youtube Videos

Typing > youtube: https://youtu.be/XfoYk_Z5AkI will render this:

Boxes / Callouts

Typing > Warning: There will be no second warning! will render this:

Warning: There will be no second warning!

Typing > Important: Pay attention! It's important. will render this:

Important: Pay attention! It’s important.

Typing > Tip: This is my tip. will render this:

Tip: This is my tip.

Typing > Note: Take note of this. will render this:

Note: Take note of this.

Typing > Note: A doc link to [an example website: fast.ai](https://www.fast.ai/) should also work fine. will render in the docs:

Note: A doc link to an example website: fast.ai should also work fine.

Footnotes

You can have footnotes in notebooks, however the syntax is different compared to markdown documents. This guide provides more detail about this syntax, which looks like this:

For example, here is a footnote {% fn 1 %}.
And another {% fn 2 %}
{{ 'This is the footnote.' | fndetail: 1 }}
{{ 'This is the other footnote. You can even have a [link](www.github.com)!' | fndetail: 2 }}

For example, here is a footnote ¹.

And another ²

1. This is the footnote.↩

2. This is the other footnote. You can even have a link!↩

	Title	Worldwide_Gross	Production_Budget	Distributor	MPAA_Rating	IMDB_Rating	Rotten_Tomatoes_Rating
0	The Land Girls	146083.0	8000000.0	Gramercy	R	6.1	NaN
1	First Love, Last Rites	10876.0	300000.0	Strand	R	6.9	NaN
2	I Married a Strange Person	203134.0	250000.0	Lionsgate	None	6.8	NaN
3	Let's Talk About Sex	373615.0	300000.0	Fine Line	None	NaN	13.0
4	Slam	1087521.0	1000000.0	Trimark	R	3.4	62.0

Ashutosh Sanzgiri’s Fastpages Blog

Setup

Run test example using model trained on synthetic data

Explore various ways to convert HDR to LDR

Comparisons

PFSTools

0

imgs/00000.png

1

output_hdrs/00000_hdr_pfs.png

ImageMagick with Autotone is the best choice!

0

imgs/00000.png

1

output_hdrs/00000_hdr_im.png

OpenCV options

0

imgs/00000.png

1

output_hdrs/00000_hdr_cv_d.png

0

imgs/00000.png

1

output_hdrs/00000_hdr_cv_m.png

0

imgs/00000.png

1

output_hdrs/00000_hdr_cv_r.png

0

imgs/00000.png

1

output_hdrs/00000_hdr_cv_u.png

Run test example on model trained with Real + Synthetic data

0

imgs/00000.png

1

output_hdrs2/00000_hdr.png

Upload your own file now

0

uploads/landscape.png

1

output_hdrs/landscape_hdr_im.png

0

uploads/landscape.png

1

output_hdrs/landscape_hdr_pfs.png

0

uploads/landscape.png

1

output_hdrs/landscape_hdr_cv_D.png

0

uploads/landscape.png

1

output_hdrs/landscape_hdr_cv_M.png

0

uploads/landscape.png

1

output_hdrs/landscape_hdr_cv_R.png

0

uploads/landscape.png

1

output_hdrs/landscape_hdr_cv_U.png

Home Automation with a Raspberry Pi

Creating auto-updating notebooks and blog posts with FastPages

References

Fastpages Notebook Blog Post

About

Front Matter

Markdown Shortcuts

Interactive Charts With Altair

Example 1: DropDown

Example 2: Tooltips

Example 3: More Tooltips

Data Tables

Images

Local Images

Remote Images

Animated Gifs

Captions

Other Elements