Lewis Acid and Base Catalysis of YNbO4 Toward Aqueous-Phase Conversion of Hexose and Triose Sugars to Lactic Acid in Water

Article abstract of DOI:10.1002/cctc.201901435

Amphoteric YNbO₄ was synthesized by the simple coprecipitation using (NH₄)₃[Nb(O₂)₄] and Y(NO₃)₃, and examined as a new solid acid-base bifunctional catalyst for various reactions including aqueous-phase conversion of glucose to lactic acid. After drying the white precipitate at 353 K for 3 h, the resultant oxide is an amorphous YNbO₄ with high densities of Lewis acid sites (0.18 mmol g^?1) and base sites (0.38 mmol g^?1). Negatively-charged lattice oxygen of amorphous YNbO₄ functioned as Lewis base sites that promote a Claisen-Schmidt-type condensation reaction with acetylacetone and benzaldehyde with comparable activity to reference catalysts. Amorphous YNbO₄ can also be applicable to the production of lactic acid from glucose in water, which gives relatively high yields (19.6 %) compared with other reference catalysts. Mechanistic studies using glucose-1-d and ²H nuclear magnetic resonance spectroscopy (NMR) revealed that YNbO₄ first converts glucose to two carbohydrates (glyceraldehyde and pyruvaldehyde) through dehydration via the formation of 3-deoxyglucosone and subsequent retro-aldolization, and these intermediates are then converted to lactic acid by both dehydration and isomerization through hydride transfer.

Full text of DOI:10.1002/cctc.201901435

Technical Note: Real-time 3D MRI in the presence of motion for MRI-guided
radiotherapy: 3D Dynamic keyhole imaging with super-resolution
a)
Taeho Kim , and Justin C. Park
Department of Radiation Oncology, Washington University School of Medicine, St Louis, MO 63110, USA
H. Michael Gach
Department of Radiation Oncology, Washington University School of Medicine, St Louis, MO 63110, USA
Department of Radiology and Biomedical Engineering, Washington University in St. Louis, St Louis, MO 63110, USA
Jaehee Chun
Department of Radiation Oncology, Yonsei University College of Medicine, Seoul 03722, South Korea
Sasa Mutic
Department of Radiation Oncology, Washington University School of Medicine, St Louis, MO 63110, USA
(
Received 22 April 2019; revised 21 June 2019; accepted for publication 22 July 2019;
published 27 August 2019)
Purpose: The purpose of this study was to present real-time three-dimensional (3D) magnetic reso-
nance imaging (MRI) in the presence of motion for MRI-guided radiotherapy (MRgRT) using
dynamic keyhole imaging for high-temporal acquisition and super-resolution generative (SRG) model
for high-spatial reconstruction.
Method: We propose a unique real-time 3D MRI technique by combining a data sharing technique
(3D dynamic keyhole imaging) with a SRG model using cascaded deep learning technique. 3D
dynamic keyhole imaging utilizes the data sharing mechanism by combining keyhole central k-space
data acquired in real-time with high-spatial, high-temporal resolution prior peripheral k-space data at
various motion positions prepared by the SRG model. The efficacy of the 3D dynamic keyhole imag-
ing with super-resolution (SR_dKeyhole) was compared to the ground-truth super-resolution images
with the original full k-space data. It was also compared with the zero-filling reconstruction (zero-
filling), conventional keyhole reconstruction with low-spatial high-temporal prior data (LR_cKey-
hole), and conventional keyhole reconstruction with super-resolution prior data (SR_cKeyhole).
3
Results: High-spatial, high-temporal resolution 3D MRI datasets (1.5 9 1.5 9 6 mm ) were gener-
3
ated from low-spatial, high-temporal resolution 3D MRI datasets (6 9 6 9 6 mm ) using the cas-
caded deep learning SRG framework (<100 ms/volume). 3D dynamic keyhole imaging with the SRG
3
model provided high-spatial, high-temporal resolution images (1.5 9 1.5 9 6 mm , 455 ms) with
the highest similarity to the ground-truth SR images without any noticeable artifacts. Structural simi-
larity indices (SSIM) of the reconstructed 3D MRI to the original SR 3D MRI were 0.65, 0.66, 0.86,
and 0.89 for zero-filling, LR_cKeyhole, SR_cKeyhole, and SR_dKeyhole, respectively (1 for identi-
cal image). In addition, average value of image relative error (IRE) of the reconstructed 3D MRI to
the original SR 3D MRI were 0.169, 0.191, 0.079, and 0.067 for zero-filling, LR_cKeyhole,
SR_cKeyhole, and SR_dKeyhole, respectively (0 for identical image).
Conclusions: We demonstrated that high-spatial, high-temporal resolution 3D MRI was feasible by
combing 3D dynamic keyhole imaging with a SRG model in terms of image quality and imaging
time. The proposed technique can be utilized for real-time 3D MRgRT. © 2019 American Associa-
tion of Physicists in Medicine [https://doi.org/10.1002/mp.13748]
Key words: dynamic keyhole, integrated MRI and radiotherapy system, real-time 3D MRI, super-
resolution
2
,9,10
1
. INTRODUCTION
shape,
since MRI produces superior soft tissue contrast
of patient anatomy without ionizing radiation. MRI-guided
radiotherapy (MRgRT) gives excellent local control with
Integrated magnetic resonance imaging (MRI) and radio-
therapy systems are clinically established in image-guided
11,12
little toxicity.
Respiratory-related tumor motion can be detected using
1
–4
radiotherapy (IGRT).
Furthermore, there are several sys-
13–16
tems with unique configurations such as inline orientation
with biplanar magnet and inline/perpendicular orientation
time-resolved 4D MRI and cine MRI.
Time-resolved 4D
MRI provides high-spatial resolution 3D MRI datasets with
multiple respiratory amplitudes/phases. For example, in
motion studies, iterative reconstruction techniques in 4D
MRI improved image quality while reducing motion
5
–8
with open magnet under development.
The integrated
MRI and radiotherapy systems can provide real-time tumor
tracking, inclusive of changes in tumor position and
4
631
Med. Phys. 46 (10), October 2019
0094-2405/2019/46(10)/4631/8
© 2019 American Association of Physicists in Medicine
4631

4
632
Kim et al.: 3D Dynamic keyhole with super-resolution
7–19
4632
1
artifacts.
However, long acquisition and retrospective
the amount of temporal k-space data acquired in real-time
(called keyhole data) and high-spatial resolution imaging is
achieved by utilizing prior peripheral k-space data in conven-
tional Fourier transform image reconstruction. By utilizing
anatomical motion information such as respiratory displace-
ment in keyhole imaging, real-time 2D MRI in the presence
of respiratory motion (i.e., dynamic keyhole imaging) can be
reconstruction times (>1 min) preclude their use for real-time
tumor tracking in MRgRT. In addition, several real-time 3D
MRI techniques were proposed with undersampled acquisi-
2
5
2
0,21
tion schemes,
but their long reconstruction times are not
suitable for real-time MRgRT. Near real-time 2D cine MRI
with frame rates (~5 frames/sec) is currently utilized in real-
time MRgRT and the acquired spatial resolution has been
compromised (0.35 9 0.35 cm ). However, tumors near the
diaphragm showed 3D motion, reaching up to a 5 cm in
2
6,27
acquired without noticeable motion artifacts.
The previ-
2
2
ous studies demonstrated that the dynamic keyhole 2D cine
imaging provided accurate tumor motion quantification with
higher frame rates (~15 frames/sec), implying a promising
2
2
23
22,24
26
range and rotating up to 45° during respiration.
Therefore, high-spatial, high-temporal resolution 3D MRI is
highly needed in real-time MRgRT.
One of the high-temporal resolution 3D MRI approaches
for real-time tumor tracking is a data sharing technique. In
conventional data sharing techniques such as keyhole imag-
ing, high-temporal resolution is accomplished by reducing
technique for real-time 3D cine MRI. However, high-spatial
resolution 3D dynamic keyhole imaging has been challenged,
since the spatial resolution of the temporal images is deter-
mined by the spatial resolution of the prior data in the
dynamic keyhole imaging in Fig. 1. In [Fig. 1(b)], high-spatial
resolution prior data with low-temporal resolution can be
Prior k-space data
Temporal k-space data
(a) dKeyhole 3D MRI:
Low-spatial
(
a)
High-temporal resolution
*
Blurring
Low-spatial
high-temporal
prior data
Low-spatial
high-temporal
keyhole data
Prior k-space data
Temporal k-space data
Full k-space data / Image
Prior k-space data
Temporal k-space data
(b) dKeyhole 3D MRI:
High-spatial
High-temporal resolution
(
b)
*Motion artifacts
High-spatial
high-temporal
keyhole data
High-spatial
low-temporal
prior data
Prior k-space data
Temporal k-space data
(
c) dKeyhole 3D MRI:
High-spatial
High-temporal resolution
(
c)
High-spatial
high-temporal
prior data
SR
Low-spatial
high-temporal
prior data
High-spatial
high-temporal
keyhole data
FIG. 1. Schematic of dynamic keyhole imaging for 3D MRI. In conventional 3D dynamic keyhole imaging in the presence of motion, (a) low-spatial, high-tempo-
ral resolution imaging is acquired (combining keyhole central k-space data in temporal data with peripheral k-space data in prior data). Since the spatial resolution
of the temporal images is determined by the spatial resolution of the prior data in the dynamic keyhole imaging, (b) high-spatial, low-temporal resolution prior
data cause motion artifacts in reconstructed high-spatial, high-temporal resolution 3D images. (c) high-spatial, high-temporal prior data generated from low-spa-
tial, high-temporal prior data by deep learning is used for the high-spatial, high-temporal 3D dynamic keyhole imaging. [Color figure can be viewed at wileyon
linelibrary.com]
Medical Physics, 46 (10), October 2019

4
633
Kim et al.: 3D Dynamic keyhole with super-resolution
4633
used in the reconstruction, but substantial blurring motion
artifacts will degrade image quality. Therefore, the high-spa-
tial, low-temporal resolution 3D prior data approach under-
performs high-spatial, high-temporal resolution 3D imaging.
Recently, several super-resolution approaches have been
A cascaded DL-based SRG model used a splitting SR gen-
eration process to make a SR model robust compared to the
conventional DL-based methods. The cascaded DL-based SR
consisted of three training stages: (a) training of a denoising
autoencoder (DAE) using noise reduced low-spatial resolu-
tion data; (b) training of a downsampling network (DSN)
using a paired low-spatial resolution (LR)/high-spatial resolu-
tion (HR) data that allows the generation of perfectly paired
LR/HR data from the clinically abundant HR scans; and (c)
the training of a SRG trained with data generated by the DSN
that maps from LR inputs to HR outputs. The training was
performed using a GeForce GTX 1080 Ti GPU (NVIDIA,
Santa Clara, CA).
2
8,29
introduced to MRI.
As post-processing means, they can
generate images with high-SNR and high-spatial resolution
within a reasonable processing time frame from low-SNR
2
9
and low-spatial images obtained with high frame rates.
Utilizing super-resolution can overcome a spatial resolution
limitation in 3D dynamic keyhole imaging. The aim of this
paper is to propose high spatial and temporal resolution 3D
MRIs in the presence of motion for real-time 3D MRgRT by
combining 3D dynamic keyhole imaging with super-resolu-
tion generative (SRG) model.
1. Training of a denoising autoencoder (DAE): Image
denosing is critical since SR generation suffers from
image noise. Conventional image denoising techniques
are not efficient in processing time and computation
power. In the training process, we used a convolutional
neural network (CNN)-based DAE for high accuracy.
The DAE was trained with pairs of noisy and denoised
LR MRIs that were preprocessed using the nonlocal
means filter (NLM). Four hundred and eighty LR
breath-hold images were used in the DAE training
2
. METHODS
The proposed techniques utilized two key components for
high-spatial, high-temporal 3D MRI: (a) dynamic keyhole
imaging for high-temporal acquisition; and (b) SRG model
for high-spatial reconstruction.
2
.A. Low-spatial, high-temporal resolution image
(training time: around 35 min).
acquisition
2
. Training of a downsampling network (DSN): After the
DAE training, paired LR and HR MR images were
used in the downsampling network training. The
images were scanned from a phantom and volunteers
in a single breath-hold. We manually selected the data
pairs until the HR and LR scans were perfectly paired.
We designed a CNN-based DSN for the downsampling
process using 480 data pairs of LR and HR images. In
the training, we ensured that the resulting LR images
were similar to the ground-truth LR images from the
All data used in this study were acquired from a 0.35 T
MRgRT system (ViewRay Inc., Oakwood Village, Ohio) with
a three head Co-60 radiation therapy delivery system. MRIs
of human subjects were acquired using anterior and posterior
torso phased array receive only coils. Twenty real-time volu-
metric 3D MRI datasets were acquired continuously (free
breathing) across multiple respiratory cycles for four human
subjects. 3D images in coronal orientation were acquired
using the true fast imaging using steady-state precession
(
TrueFISP). The generalized auto calibrating partially parallel
scanner. HR MR images of 256 9 256 pixels
2
acquisition (GRAPPA) technique was used to speed up image
acquisition (iPAT = 2). Imaging parameters are base resolu-
(
6
1.5 9 1.5 mm ) and the corresponding LR images of
2
4 9 64 pixels (6.0 9 6.0 mm ) were used in the
testing and inferencing steps (training time: around
0 min).
tion = 64, phase partial Fourier = 6/8, TR/TE = 1.75/
3
0
.81 ms, voxel size = 6 9 6 9 6 mm , and total acquisition
2
time/volume = 420 ms.
3
. Training of SRG model: The robustness of the DSN
was a key component in the performance of the SRG
model because the DNS was used to generate LR MRIs
for the SRG model training. The SRG network was
then trained with 4,475 data pairs from the set of axial
LR MRIs to create axial SR MRIs and 1,375 data pairs
from LR 4D MRIs to create SR 4D MRIs, respectively.
The data pairs consisted of clinically available HR MR
images and the corresponding LR images generated by
the DSN. In the training, we verified the SRG model
with fast axial MRI in a breath-hold and multivolumet-
ric cine 3D MRI. The axial SR MRIs were generated
from the set of axial LR MRIs acquired in a short
breath-hold interval (<3 s). SR multivolumetric cine
2
.B. High-spatial, high-temporal resolution prior
image generation (Super Resolution) using
Cascaded deep learning
The deep learning (DL)-based SR generative (SRG)
model for MRI accomplishes a significant improvement in
spatial resolution compared to conventional SR techniques
3
0
such as interpolation-based methods. SR images can be
generated without transformation models since DL-based
methods utilize direct mapping based on information
extracted from previous datasets. Recent studies have
shown that the in-plane resolution of physically scanned
low-spatial resolution 3D MRI can be improved to a four
times higher resolution without compromising the image
3D MRIs were generated from multivolumetric LR
cine 3D MRI acquired at rate of 2 volumes/sec (train-
ing time: around 4 h 15 min).
2
9
quality.
Medical Physics, 46 (10), October 2019

4
634
Kim et al.: 3D Dynamic keyhole with super-resolution
4634
After the verification of DL-based SRG model, we gener-
keyhole reconstruction with super-resolution prior data
(SR_cKeyhole) using MATLAB R2018b (The MathWorks,
Natick, USA). All reconstruction techniques used high-spa-
tial resolution central k-space data.
ated the SR multivolumetric cine 3D MRIs from LR MRIs
using the SRG model (20 SR volumetric cine 3D MRI data-
sets/each human subject). In this study, we utilized the pairs
of the SR multivolumetric cine 3D MRI and the LR multivol-
umetric cine 3D MRI. The SR MRIs were used as the
ground-truth images for the comparison study.
The quality of the reconstructed images with each of
four methods was compared with the ground-truth SR
images using the original full k-space data. Image quality
and artifacts were evaluated using: (a) image intensity dif-
ference, (b) structural similarity index (MATLAB function:
2
.C. 3D dynamic keyhole imaging with a library of
the prior data
31
SSIM), and (c) average value of image relative error
18
(
IRE). SSIM assesses images in terms of luminance, con-
Figure 1 shows the scheme of the 3D dynamic keyhole
imaging. The dynamic keyhole imaging uses a library of
prior peripheral k-space data in combination with keyhole k-
space data acquired in real-time. For high-spatial, high-tem-
poral 3D dynamic keyhole imaging, high spatial and tempo-
ral resolution prior k-space data at various respiratory
positions were needed as a library of the prior data in 3D
dynamic keyhole imaging.
Twenty real-time volumetric 3D MRI datasets from each
of four human subject were used in the comparison study.
Each dataset included low-spatial, high-temporal resolution
images from the acquisition and high-spatial, high-temporal
resolution images generated using the SRG model. Respira-
tory displacement was extracted from the profile of the dia-
phragm on the coronal middle slice of the high-spatial, high-
temporal resolution 3D MRI. Based on each subject’s respira-
tory patterns, approximately ten respiratory positions were
sampled through a couple of respiratory cycles. The corre-
sponding high-spatial high-temporal resolution MRI datasets
were used as the prior data associated with the respiratory
position. The remaining high-spatial, high-temporal resolu-
tion MRI datasets were used as the temporal data in the 3D
dynamic keyhole imaging.
trast, and structural comparisons; a SSIM value of 1 indi-
cates that the reconstructed image and the original image
are identical.
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
PÂ
Ã
2
Ioriginalðx; y; zÞ ꢀ Ireconðx; y; zÞ
x;y;z
IRE ¼
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
(1)
PÂ
Ã
2
I
ðx; y; zÞ
original
x;y;z
where Ioriginal is the original image and Irecon is the recon-
structed image in Eq. (1). An IRE value of 0 indicates that
the reconstructed image and the original image are identical.
3. RESULTS
3
.A. High-spatial, high-temporal resolution image
generation for 3D dynamic keyhole imaging
High-spatial, high-temporal resolution 3D MRI datasets
(image matrix: 256 9 256 9 10, spatial resolution:
3
1.5 9 1.5 9 6 mm ) were generated from low-spatial, high-
temporal resolution 3D MRI datasets acquired in the coronal
plane (image matrix: 64 9 64 9 10, spatial resolution:
3
In the simulation studies, all SR image data were Fourier
transformed into k-space. Approximately ten volumetric SR
MR k-space datasets were used as the prior k-space dataset
and the remaining volumetric SR MR k-space datasets were
used as the temporal k-space datasets. The size of the keyhole
k-space data was empirically determined in consideration of
image quality and time. The k-space data matrix (kx x ky x
kz) was 256 9 256 9 10. The size of keyhole k-space data
was 26 ky lines/slice. The size of the prior peripheral k-space
data was 230 ky lines/slice. The prior peripheral and keyhole
k-space data were matched based on respiratory displace-
ments to accelerate acquisition time and improve image qual-
6 9 6 9 6 mm , acquisition time: 420 ms/volume) by using
the cascaded deep learning framework (processing time:
<100 ms/volume). Super resolution images were inverse
Fourier transformed which were used as high-spatial, high-
temporal resolution prior k-space data in 3D dynamic key-
hole imaging. Figure 2 shows both the low-spatial resolution
(low-spatial resolution: LR) and high-spatial resolution SR
(high-spatial resolution: HR) images from four human sub-
jects.
3
.B. High-spatial, high-temporal resolution 3D MRI
and comparison
2
6
ity in the presence of motion. Images were reconstructed
using the conventional 3D Fourier transform with the com-
bined full k-space dataset.
Figure 3 shows the high-spatial resolution original image
with full k-space data (estimated acquisition time: 1.75 ms/
TR 9 256 ky lines 9 10 kz lines = 4,480 ms) and the
reconstructed high-spatial resolution images of four tech-
niques with the keyhole k-space data (26 ky
2
.D. Comparison studies and evaluation
The efficacy of the 3D dynamic keyhole imaging with
lines/slices,
estimated
acquisition
time:
1.75 ms/
super-resolution (SR_dKeyhole) was compared with zero-
filling reconstruction (zero-filled in the k-space data periph-
ery), conventional keyhole reconstruction with low-spatial,
high-temporal prior data (LR_cKeyhole), and conventional
TR 9 26 ky lines 9 10 kz lines = 455 ms). The zero-fill-
ing reconstruction resulted in blurred images due to a lack
of fine image details from missing peripheral k-space data
(zero-filled). The LR_cKeyhole reconstruction resulted in
Medical Physics, 46 (10), October 2019

4
635
Kim et al.: 3D Dynamic keyhole with super-resolution
4635
I-LR
II-LR
III-LR
IV-LR
IV-HR
(
(
a) Low-spatial resolution (LR)
I-HR
II-HR
III-HR
b) High-spatial resolution (HR)
3
FIG. 2. Top row: (a) Low-spatial resolution (LR) data with image matrix: 64 9 64 9 10, spatial resolution: 6 9 6 9 6 mm , acquisition time: 420 ms/volume.
Bottom row: (b) Output generated from cascaded deep learning framework, high-spatial resolution (HR) data with image matrix: 256 9 256 9 10, spatial reso-
3
lution: 1.5 9 1.5 9 6 mm . Generated high-spatial, high-temporal resolution prior images were inverse Fourier transformed which were used in 3D dynamic
keyhole imaging. I-VI indicate human subject numbers.
blurred images due to the low-spatial resolution prior data
with unmatched motion information. SR_cKeyhole recon-
struction improved image quality by using a high-spatial
resolution prior dataset, but motion-related artifacts were
still present (white arrow) due to utilizing a single prior k-
space dataset. SR_dKeyhole reconstructed images were
similar to the original SR images since SR_dKeyhole
combined high-spatial resolution prior data with motion
information. Also, the difference maps of the reconstructed
coronal images from the original high-spatial resolution
data indicated that SR_dKeyhole was the most favorable
technique for real-time high-spatial resolution 3D MRI.
The differences of the images are displayed with an inten-
sity range between 0 and 9000 for visual inspection.
Structural similarity indices of the reconstructed 3D MRI
to the original 3D MRI were 0.65, 0.66, 0.86, and 0.89
for zero-filling, LR_cKeyhole, SR_cKeyhole, and
SR_dKeyhole, respectively (1 for identical image). In addi-
tion, the IRE scores from comparing the reconstructed 3D
MRI to the original 3D MRI were 0.169, 0.191, 0.079,
and 0.067 for zero-filling, LR_cKeyhole, SR_cKeyhole,
and SR_dKeyhole, respectively.
Figure 4 shows sagittal and axial images of four imag-
ing techniques. Image planes are indicated on the coronal
image (s: sagittal imaging plane, ax: axial imaging plane):
(a) original SR image, (b) zero-filling image, (c) conven-
tional keyhole image with low-spatial resolution prior data
(LR_cKeyhole), (d) conventional keyhole image with super-
spatial resolution prior data (SR_cKeyhole), and (e)
dynamic keyhole image with super-spatial resolution prior
data (SR_dKeyhole), respectively. Since the super-resolu-
tion was conducted in the coronal imaging plane, the
Original
Zero-filling
LR_cKeyhole
SR_cKeyhole
SR_dKeyhole
Difference
Range
[0 -9000]
SSIM = 0.65
IRE = 0.164
SSIM = 0.63
IRE = 0.197
SSIM = 0.75
IRE = 0.119
SSIM = 0.88
IRE = 0.072
FIG. 3. Comparison of four reconstructed images (estimated acquisition time: 455 ms) to the original image (estimated acquisition time: 4,480 ms): (from left to
right) images with full k-space data, zero-filling, conventional keyhole with low-spatial resolution prior data (LR_cKeyhole), conventional keyhole with super-
spatial resolution prior data (SR_cKeyhole), and dynamic keyhole with super-spatial resolution prior data (SR_dKeyhole). Top row: Reconstructed coronal
images of human subject-I. Bottom row: Image difference between the reconstructed image with the keyhole k-space data and the original image with full k-space
data. SSIM: structural similarity index. IRE: average value of image relative error. The white arrow in SR_cKeyhole indicates motion-related artifacts.
Medical Physics, 46 (10), October 2019

4
636
Kim et al.: 3D Dynamic keyhole with super-resolution
4636
Sagittal
ax
ax-(a)
ax-(b)
ax-(c)
ax-(d)
ax-(e)
s
Coronal
s-(a) s-(b) s-(c) s-(d) s-(e)
FIG. 4. Sagittal and axial images are presented. Image planes are indicated on the coronal image. (a) Original image, (b) zero-filling, (c) conventional keyhole
with low-spatial resolution prior data (LR_cKeyhole), (d) conventional keyhole with super-spatial resolution prior data (SR_cKeyhole), and (e) dynamic keyhole
with super-spatial resolution prior data (SR_dKeyhole). s indicates sagittal imaging plane and ax indicates axial imaging plane.
anterior–posterior (AP) direction of the sagittal image and
4
. DISCUSSION
the axial image showed less distinction among four recon-
structed images from the original image. According to the
visual inspection, SR_dKeyhole reconstructed images were
still most similar to the original SR images without any
distinct artifacts.
In MRgRT, high-spatial volumetric 3D MRI is used for
patient alignment and target/organ delineations. During treat-
ment delivery, high-temporal 2D cine MRI (~ 5 frames/sec)
or low-temporal 3D MRI (1 volume/7 s) may be used for
2
3
Structural similarity indices are presented in Fig. 5. In the
box plot, 37 respiratory samples were included: ten respira-
tory samples from each human subject I-III and seven respira-
tory samples from human subject IV (due to long respiratory
cycle). Reconstructed 3D images of four human subjects
show different level of similarity to the original 3D MRI;
SSIM mean ꢁ SD were 0.65 ꢁ 0.02, 0.66 ꢁ 0.03,
continuous patient monitoring. However, since tumors near
the diaphragm showed 3D motion, both of the approaches
cannot properly detect 3D tumor motion in real-time. There-
fore, high-spatial, high-temporal resolution 3D MRI is highly
needed in real-time MRgRT.
We demonstrated that high-spatial, high-temporal 3D MRI
was feasible by combining 3D dynamic keyhole imaging with
the SRG model. High-spatial resolution images were pre-
pared as the prior data using super-resolution prior data
(<100 ms/volume) generated from low-spatial high-temporal
resolution images (< 1 min for 20 volumes). 3D dynamic
keyhole imaging accelerated the acquisition time by reducing
the size of the keyhole data acquired in real-time, resulting in
high-temporal resolution imaging. According to our findings,
the proposed technique (estimated acquisition time: 455 ms)
provided image quality close to the ground-truth SR image
with original full k-space data (estimated acquisition time:
4,480 ms). In addition, the proposed technique used the
0
.86 ꢁ 0.07, and 0.89 ꢁ 0.05 for zero-filling, LR_cKeyhole,
SR_cKeyhole, and SR_dKeyhole, respectively. In addition,
IRE mean ꢁ SD were 0.169 ꢁ 0.017, 0.191 ꢁ 0.025,
0
.079 ꢁ 0.024, and 0.067 ꢁ 0.016 for zero-filling,
LR_cKeyhole, SR_cKeyhole, and SR_dKeyhole, respec-
tively. Reconstructed images by SR_dKeyhole show the high-
est similarity and the least variance to the original images,
while SR_cKeyhole show the second highest similarity but
the largest variance to the original images. In contrast,
LR_cKeyhole images show the least similarity to the
original images.
SSIM
IRE
1
0.25
(
a)
(b)
0
0
0
0
0
.95
.9
.85
.8
.75
.7
.65
.6
.55
.5
0
0.2
0
0.15
0.1
0.05
0
0
0
0
Imaging
Imaging
FIG. 5. Box plot of (a) SSIM and (b) IRE in similarity evaluation between the original 3D images and the reconstructed 3D images with four imaging techniques.
a) Mean values of SSIM were 0.65, 0.66, 0.86, and 0.89 for zero-filling, LR_cKeyhole, SR_cKeyhole, and SR_dKeyhole. In addition, (b) mean values of IRE
(
are 0.169, 0.191, 0.079, and 0.067 for zero-filling, LR_cKeyhole, SR_cKeyhole, and SR_dKeyhole, respectively. IRE, image relative error; SSIM, Structural simi-
larity indices. [Color figure can be viewed at wileyonlinelibrary.com]
Medical Physics, 46 (10), October 2019

4
637
Kim et al.: 3D Dynamic keyhole with super-resolution
4637
conventional Fourier transform image reconstruction, so no
additional computational time and power were needed, unlike
iterative reconstructions of real-time 3D MRI with undersam-
based delineations, and plan verification. The acquisition
dead time can be used for the prior data preparation including
the low-spatial, high-temporal imaging (<1 min) and the cor-
responding SR image generation (<1 min). Once the treat-
ment starts, 3D dynamic keyhole imaging can be launched
for real-time 3D MRgRT.
17–21
pled schemes and retrospective 4D MRI.
Combing paral-
lel, partial Fourier acquisition and/or reduced field of view
approaches with 3D dynamic keyhole imaging may further
accelerate the 3D MRI imaging time.
This study has a few limitations. Although four cases
showed a promising results based on visual inspection, SSIM
and IRE of the reconstructed images, more cases with differ-
ent levels of motion irregularity may examine the stability of
the proposed technique. Second, super-resolution was per-
formed in the coronal imaging plane (in-plane: superior–infe-
rior and left–right) and the through-plane (anterior–posterior)
resolution remained (6 mm). The data reduction in 3D
dynamic keyhole imaging was only along ky. The data reduc-
tion of ky and kz in 3D dynamic keyhole imaging can be
studied once the super-resolution conducts in both in-plane
and through-plane. Third, the proposed technique has not
been tested in the commercial MRIs. All findings were based
on off-line simulation. Fourth, 3D image-based segmentation
and gating techniques were not studied in this report.
In terms of the imaging time, the conventional keyhole
imaging with low spatial-resolution prior data (LR_cKey-
hole) is feasible without the support of the SRG model.
The image quality appears better than the zero-filled recon-
structed images. However, SSIM and IRE show that the
image similarity of the conventional keyhole images is
quite low compared to SR_cKeyhole and SR_dKeyhole,
supporting the necessity of high-spatial resolution prior
data in keyhole imaging. Also, the SRG model can be
directly used to generate high-spatial, high-temporal 3D
MRI with higher power computation support. Compared to
the direct use of the SRG model, the dynamic keyhole
imaging utilized higher input data fidelity, providing a sim-
ple and robust imaging method for real-time MRgRT. For
example, the SRG model utilizes low-spatial resolution
images (image matrix: 64 9 64 from k-space data matrix
with 24 phase encodes using iPAT = 2 and 6/8 partial
Fourier). In contrast, the dynamic keyhole reconstruction
uses partial high-spatial resolution central k-space data (k-
space data matrix: 256 9 26) in its reconstruction. In addi-
tion, the input data fidelity can be increased by updating
partial high-spatial resolution peripheral k-space data
through temporal data acquisition in the dynamic keyhole
imaging. Therefore, higher input data fidelity is a key com-
ponent of the dynamic keyhole imaging compared to the
SRG model approach.
5
. CONCLUSION
We demonstrated that high-spatial, high-temporal 3D MRI
was feasible by combing 3D dynamic keyhole imaging with
the SRG model. The technique’s image quality was similar to
the original SR images with full k-space data while achieving
large accelerations in acquisition time. After the library
preparation, there was no additional computational time and
power needed with the conventional Fourier transform image
reconstruction in 3D imaging unlike undersampled
approaches. Validating and evaluating robustness of the tech-
nique are ongoing. Based on our findings, the proposed 3D
dynamic keyhole imaging combined with the SRG model is a
promising technique for real-time MRgRT, which will estab-
lish a paradigm for real-time 3D imaging scheme, structure
segmentation, and gating algorithm.
By utilizing anatomical motion information such as respi-
ratory displacement in keyhole imaging, real-time MRI in
the presence of respiratory motion can be acquired without
substantial motion artifacts. Figures 3 and 5 show the limita-
tion of the conventional keyhole imaging including distinct
motion artifacts and similarity variation depending on the
respiratory displacement. In Fig. 5, SR_dKeyhole recon-
structed images show the highest similarity and the least
variance to the original images. SR_cKeyhole show the sec-
ond highest similarity but the largest variance to the original
images due to uncoupling of the prior peripheral k-space
data from the temporal central k-space data, thus indicating
the importance of the respiratory phase information. In this
simulation study, respiratory displacement was extracted
from the profile of the diaphragm on the middle coronal
slice of the 3D MRI. In MRI scanners, the respiratory signal
can be extracted as a 1D Fourier transform of the kx line at
ACKNOWLEDGMENTS
Provisional patent “Methods and Systems for Real-Time
3D MRI” on file. Dr. Mutic has consulted for ViewRay and
Varian. Washington University in St. Louis has a master
research agreement and receives research funding from View-
Ray unrelated to this study.
CONFLICT OF INTEREST
The authors have no relevant conflicts of interest to dis-
close.
18
the ky-kz center.
In our current MRgRT workflow,
2
,3,10
there are no MRI
acquisitions occurring between 3D MRI in treatment setup
and 2D cine in treatment delivery because there are few treat-
ment preparation steps: patient alignment, daily anatomy-
a)
Author to whom correspondence should be addressed. Electronic mail:
taehokim@wustl.edu; Telephone: +314-273-3316.
Medical Physics, 46 (10), October 2019

4
638
Kim et al.: 3D Dynamic keyhole with super-resolution
4638
REFERENCES
16. Lee D, Greer PB, Ludbrook J, et al. Audiovisual biofeedback improves
cine–magnetic resonance imaging measured lung tumor motion consis-
tency. Int J Radiat Oncol Biol Phys. 2016;94:628–636.
17. Han F, Zhou Z, Cao M, Yang Y, Sheng K, Hu P. Respiratory motion-re-
solved, self-gated 4D-MRI using rotating cartesian k-space (ROCK).
Med Phys. 2017;44:1359–1368.
18. Wang C, Subashi E, Yin F-F, Chang Z, Cai J. A Spatiotemporal-Con-
strained Sorting Method for Motion-Robust 4D-MRI: a feasibility study.
Int J Radiat Oncol Biol Phys. 2019;103:758–766.
1
. Mutic S, Dempsey JF. The ViewRay system: magnetic resonance–
guided and controlled radiotherapy. Semin Radiat Oncol. 2014;24:196–
99.
1
2. Green OL, Rankine LJ, Cai B, et al. First clinical implementation of
real-time, real anatomy tracking and radiation beam control. Med Phys.
2018;45:3728–3740.
3
. Raaymakers B, J €u rgenliemk-Schulz I, Bol G, et al. First patients treated
with a 1.5 T MRI-Linac: clinical proof of concept of a high-precision,
high-field MRI guided radiotherapy treatment. Phys Med Biol. 2017;62:
L41–L50.
19. Deng Z, Pang J, Yang W, et al. Four-dimensional MRI using three-di-
mensional radial sampling with respiratory self-gating to characterize
temporal phase-resolved respiratory motion in the abdomen. Magn
Reson Med. 2016;75:1574–1585.
4
5
6
. Henke L, Contreras J, Green O, et al. Magnetic resonance image-guided
radiotherapy (MRIgRT): A 4.5-year clinical experience. Clin Oncol (R
Coll Radiol). 2018;30:720–727.
. Jia X, Tian Z, Xi Y, Jiang SB, Wang G. New concept on an integrated
interior magnetic resonance imaging and medical linear accelerator sys-
tem for radiation therapy. J Med Imaging. 2017;4:015004.
. Keall PJ, Barton M, Crozier S. The Australian magnetic resonance imag-
ing–linac program. Semin Radiat Oncol. 2014;24:203–206.
2
0. Kim YC, Lebel RM, Wu Z, Ward SLD, Khoo MC, Nayak KS. Real-time
3
D magnetic resonance imaging of the pharyngeal airway in sleep
apnea. Magn Reson Med. 2014;71:1501–1510.
1. Uecker M, Zhang S, Voit D, Dietmar K, Frahm J. Real-time MRI: recent
2
advances using radial FLASH. Imaging Med. 2012;4:461–476.
2. Suh Y, Dieterich S, Cho B, Keall PJ. An analysis of thoracic and abdom-
inal tumour motion for stereotactic body radiotherapy patients. Phys
Med Biol. 2008;53:3623.
2
7. Fallone B, Murray B, Rathee S, et al. MR images obtained during mega-
voltage photon irradiation from a prototype integrated linac-MR system.
Med Phys. 2009;36:2084–2088
. Liney GP, Whelan B, Oborn B, Barton M, Keall P. MRI-linear
accelerator radiotherapy systems. Clin Oncol (R Coll Radiol). 2018;30:
23. Plathow C, Schoebinger M, Fink C, et al. Quantification of lung tumor
volume and rotation at 3D dynamic parallel MR imaging with view shar-
ing: preliminary results. Radiology. 2006;240:537–545.
24. Keall PJ, Mageras GS, Balter JM, et al. The management of respiratory
motion in radiation oncology report of AAPM Task Group 76a. Med
Phys. 2006;33:3874–3900.
8
686–691.
9
. de Koste JRvS, Palacios MA, Bruynzeel AM, Slotman BJ, Senan S,
Lagerwaard FJ. MR-guided gated stereotactic radiation therapy delivery
for lung, adrenal, and pancreatic tumors: a geometric analysis. Int J
Radiat Oncol Biol Phys. 2018;102:858–866.
2
5. Van Vaals JJ, Brummer ME, Thomas Dixon W, et al. “Keyhole” method
for accelerating imaging of contrast agent uptake. J Magn Reson Imag-
ing. 1993;3:671–675.
1
0. Werensteijn-Honingh AM, Kroon PS, Winkel D, et al. Feasibility of
2
6. Lee D, Greer PB, Pollock S, Kim T, Keall P. Quantifying the accuracy
of the tumor motion and area as a function of acceleration factor for the
simulation of the dynamic keyhole magnetic resonance imaging method.
Med Phys. 2016;43:2639–2648.
stereotactic radiotherapy using a 1.5 T MR-linac: Multi-fraction treat-
ment of pelvic lymph node oligometastases. Radiother Oncol.
2
019;134:50–54.
1. Heerkens HD, Van Vulpen M, Erickson B, et al. MRI guided stereotactic
radiotherapy for locally advanced pancreatic cancer. British J Radiol.
018;91:20170563.
2. Henke LE, Green OL, Schiff J, et al.First Reported Case of Pediatric
1
2
7. Lee D, Pollock S, Whelan B, Keall P, Kim T. Dynamic keyhole: A novel
method to improve MR images in the presence of respiratory motion for
real-time MRI. Med Phys. 2014;41:072304.
2
1
2
8. Van Reeth E, Tham IW, Tan CH, Poh CL. Super-resolution in magnetic
Radiation Treatment with Magnetic Resonance Image-Guided Radio-
therapy. Advances in Radiation Oncology. 2019;31:233–236.
3. Stemkens B, Paulson ES, Tijssen RH. Nuts and bolts of 4D-MRI for
resonance imaging:
a review. Concepts Mag Reson Part A.
2012;40:306–325.
1
29. Chun J, Zhang H, Gach HM, et al. MRI super resolution reconstruction
for MRI-guided adaptive radiotherapy using cascaded deep learning: in
the presence of limited training data and unknown translation model.
Medical physics. 2019;46:4148–4164.
30. Manj ꢀo n JV, Coup ꢀe P, Buades A, Fonov V, Collins DL, Robles M. Non-
local MRI upsampling. Med Image Anal. 2010;14:784–792.
radiotherapy. Phys Med Biol. 2018;63:21TR01.
1
4. Lee D, Kim S, Palta J, Keall BL, Kim T. A retrospective 4D-MRI based
on 2D diaphragm profiles for lung cancer patients. J Med Imaging
Radiat Oncol. 2019;63:360–369.
5. Harris W, Yin F-F, Wang C, Zhang Y, Cai J, Ren L. Accelerating volu-
metric cine MRI (VC-MRI) using undersampling for real-time 3D target
localization/tracking in radiation therapy: a feasibility study. Phys Med
Biol. 2017;63:01NT01.
1
31. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assess-
ment: from error visibility to structural similarity. IEEE Trans Image
Process. 2004;13:600–612.
Medical Physics, 46 (10), October 2019