Discovery of potent antagonists of the antiapoptotic protein XIAP for the treatment of cancer

Article abstract of DOI:10.1021/jm040037k

Inhibitor of apoptosis (IAP) proteins are overexpressed in many cancers and have been implicated in tumor growth, pathogenesis, and resistance to chemo- or radiotherapy. On the basis of the NMR structure of a SMAC peptide complexed with the BIR3 domain of X-linked IAP (XIAP), a novel series of XIAP antagonists was discovered. The most potent compounds in this series bind to the baculovirus IAP repeat 3 (BIR3) domain of XIAP with single-digit nanomolar affinity and promote cell death in several human cancer cell lines. In a MDA-MB-231 breast cancer mouse xenograft model, these XIAP antagonists inhibited the growth of tumors. Close structural analogues that showed only weak binding to the XIAP-BIR3 domain were inactive in the cellular assays and showed only marginal in vivo activity. Our results are consistent with a mechanism in which ligands for the BIR3 domain of XIAP induce apoptosis by freeing up caspases. The present study validates the BIR3 domain of XIAP as a target and supports the use of small molecule XIAP antagonists as a potential therapy for cancers that overexpress XIAP.

Full text of DOI:10.1021/jm040037k

J . Med. Chem. 2004, 47, 4417-4426
4417
Discover y of P oten t An ta gon ists of th e An tia p op totic P r otein XIAP for th e
Tr ea tm en t of Ca n cer
Thorsten K. Oost,*,† Chaohong Sun,† Robert C. Armstrong,‡ Ali-Samer Al-Assaad,‡ Stephen F. Betz,§
Thomas L. Deckwerth,‡ Hong Ding,† Steven W. Elmore,† Robert P. Meadows,# Edward T. Olejniczak,†
Andrew Oleksijew,† Tilman Oltersdorf,‡ Saul H. Rosenberg,† Alexander R. Shoemaker,† Kevin J . Tomaselli,‡
Hua Zou,‡ and Stephen W. Fesik*,†
Cancer Research, Global Pharmaceutical Research and Development, Abbott Laboratories, 100 Abbott Park Road,
Abbott Park, Illinois 60064, Idun Pharmaceuticals, 9380 J udicial Drive, San Diego, California 92121, Neurocrine Biosciences,
10555 Science Center Drive, San Diego, California 92121, and Ferring Research Institute, 3550 General Atomics Court,
San Diego, California 92121
Received February 13, 2004
Inhibitor of apoptosis (IAP) proteins are overexpressed in many cancers and have been
implicated in tumor growth, pathogenesis, and resistance to chemo- or radiotherapy. On the
basis of the NMR structure of a SMAC peptide complexed with the BIR3 domain of X-linked
IAP (XIAP), a novel series of XIAP antagonists was discovered. The most potent compounds in
this series bind to the baculovirus IAP repeat 3 (BIR3) domain of XIAP with single-digit
nanomolar affinity and promote cell death in several human cancer cell lines. In a MDA-MB-
231 breast cancer mouse xenograft model, these XIAP antagonists inhibited the growth of
tumors. Close structural analogues that showed only weak binding to the XIAP-BIR3 domain
were inactive in the cellular assays and showed only marginal in vivo activity. Our results are
consistent with a mechanism in which ligands for the BIR3 domain of XIAP induce apoptosis
by freeing up caspases. The present study validates the BIR3 domain of XIAP as a target and
supports the use of small molecule XIAP antagonists as a potential therapy for cancers that
overexpress XIAP.
In tr od u ction
executioner caspases 3 and 7,6-8 which prevents normal
substrate processing and leads to the inhibition of
apoptosis. Recently, a mammalian protein called SMAC
(also called DIABLO) was identified that triggers apo-
ptosis by abrogating the inhibitory effects of IAPs on
caspases.9,10 SMAC promotes caspase-9 activation by
competing with the caspase for the same binding groove
on the surface of the BIR3 domain. The structural basis
for the binding of SMAC to the BIR3 domain of XIAP
has been elucidated by NMR and X-ray structural
analysis.11-13 The SMAC N-terminus binds to the BIR3
domain in an extended conformation with only the first
four amino acid residues (AVPI) contacting the protein.
This tetrapeptide motif was envisioned as a good start-
ing point for the design of peptidomimetic compounds
that would bind to XIAP and induce apoptosis in a
fashion similar to the protein SMAC. Such SMAC
mimetics may be useful for treating cancers that are
resistant to proapoptotic drugs due to the overexpres-
sion of IAPs. Indeed, several groups have reported that
SMAC-derived peptides that bind to XIAP can sensitize
cancer cell lines to undergo programmed cell death.14-16
Peptides have also been shown to have an effect in an
in vivo intracranial malignant glioma xenograft model.17
A peptide comprised of the 7mer N-terminus of SMAC
linked to the shuttle peptide TAT from HIV strongly
enhanced the antitumor activity of Apo2L/TRAIL in this
in vivo model. Despite these impressive studies, the use
of a peptide from the SMAC N-terminus as a therapeu-
tic agent has several disadvantages. First, truncated
N-terminal SMAC peptides bind only weakly to the
BIR3 domain of XIAP (Kd ∼ 0.5 µM).11 Furthermore,
Apoptosis plays a critical role in the development and
homeostasis of multicellular organisms. Inappropriate
suppression or activation of apoptosis can lead to a
variety of diseases. For example, in many cancers the
cell death machinery is inhibited by the upregulation
of antiapoptotic proteins, suggesting that the restoration
of apoptotic activity might be an effective approach for
treating these cancers. Members of the inhibitor of
apoptosis (IAP) family of proteins are upregulated in
cancers1-3 and inhibit programmed cell death through
their ability to directly inhibit members of the caspase
family of apoptotic enzymes. Human X-linked IAP
(XIAP), the best-characterized among the IAPs, is
believed to directly inhibit particular caspases via its
∼70-amino acid baculovirus IAP repeat (BIR) domains.
The BIR3 domain of XIAP binds directly to the small
subunit of caspase-9,1,4 the initiator caspase in the
mitochondrial pathway of apoptosis. As evident from the
X-ray structure of the complex, XIAP sequesters
caspase-9 in a catalytically inactive monomeric state,
thus preventing formation of the catalytically active
caspase-9 homodimer.5 On the other hand, a region
containing the BIR2 domain of XIAP inhibits the
* Address correspondence to either author: Thorsten K. Oost, Abbott
GmbH & Co. KG, Knollstrasse, 67061 Ludwigshafen, Germany; ph:
+49 (0)621 589 2141; e-mail: thorsten.oost@abbott.com; Stephen W.
Fesik, Department 460, Bldg. AP10-LL, 100 Abbott Park Road, Abbott
Park, IL 60064-6098, USA; ph: 847 937 1207; e-mail: stephen.fesik@
abbott.com.
† Abbott Laboratories.
‡ Idun Pharmaceuticals.
§ Neurocrine Biosciences.
#
Ferring Research Inst.
10.1021/jm040037k CCC: $27.50 © 2004 American Chemical Society
Published on Web 07/31/2004

4418 J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18
Oost et al.
Ta ble 1. Binding Affinity to the BIR3 Domain of XIAP for
accommodated, which is in good agreement with the
three-dimensional structure of the SMAC/XIAP-BIR3
complex in which the Ala methyl group fits tightly into
the protein pocket.11 The importance of the hydrophobic
interaction between the Ala methyl group and W310 of
BIR3 is underscored by the lack of activity for the Gly
analogue. Substitution of the terminal amino group with
one methyl group was found to be tolerated without loss
of binding affinity. However, N,N-dimethyl substitution
abolishes binding. The peptide library in which position
2 was varied afforded a relatively shallow SAR profile
(Kd values between 0.04 and 0.29 µM) for L-configured
neutral and basic residues, concordant with the side
chain of residue 2 pointing away from the protein into
solution. Residues with a D-configuration or those
containing a gem-dimethyl motif (Aib) were inactive.
These results are consistent with the 3D structure of
the SMAC/XIAP-BIR3 complex in which the side chain
of a D-amino acid is predicted to clash sterically with
the protein. The relatively weak binding affinity of the
Gly analogue may be caused by a lack of conformational
rigidity, as compared with analogues that bear L-
configured residues in the second position. Replacing
the Val at position 2 with N-Me-Ala abolished binding
to the BIR3 domain, underscoring the importance of the
intermolecular hydrogen bond involving the amide of
residue 2 which stabilizes complex formation. Attempts
to replace Pro in position 3 with a variety of amino acids,
in most cases, resulted in a significant loss of binding
affinity. In the SMAC/XIAP-BIR3 complex the 4
(gamma) and 5 (delta) methylene groups of the proline
ring contact Trp 323. The contacts with the 5 (delta)
position of the proline ring cannot be mimicked by other
natural amino acids. In addition, a proline at this
position may conformationally restrict the peptide.
Interestingly, the azetidine (Aze) analogue bearing a
four- instead of a five-membered ring afforded a similar
binding affinity as the Pro analogue, suggesting a closely
related bioactive conformation. In position 4, aromatic
residues, such as Phg, Phe, homo-Phe (in this order),
or 1-naphthylalanine (1-Nal), afforded the best binding
affinities. It is interesting to note, however, that ali-
phatic residues of comparable hydrophobicity, such as
cyclohexylalanine (Cha), exhibited over a 10-fold loss
of binding affinity. This result is consistent with previ-
ous findings19 as well as with the fact that other BIR3
binding proteins, such as Reaper, Hid, Sickle and
human caspase-9 carry an aromatic residue in the
corresponding position. NMR studies on a AVPF/XIAP-
BIR3 peptide complex were consistent with a binding
mode that is similar to the SMAC-derived tetrapeptide
AVPI. In particular, NOEs from the Phe of the AVPF
peptide and Gly 306, Lys 297, and Lys 299 indicated
that the Phe of the peptide sat in the same pocket as
the parent Ile. These data support the hypothesis that
the preference for Phe is due to the additional favorable
hydrophobic interactions that arise from placing a
phenyl ring between the hydrophobic portion of the side
chains of Lys 297 and Lys 299 (Figure 1A).
Representative Examples from the Peptide Libariesa
position Kd position Kd position Kd
position
4
Kd
[µM]
1
[µM]
2
[µM]
0.06 Pro
2.7 Gly
0.29 Aib
0.09 Sar
3
[µM]
Ala
Gly
0.06 Val
0.06 Phe
3 Gly
>20 Val
0.6
0.06
3.4
0.9
0.28
0.36
0.9
0.97
0.3
0.18
0.09
0.19
0.03
9.6
0.04 Ala
0.9 Abu
Gly
Abu
Val
Leu
Ile
Nle
Cha
Nle
D-Ala
Aib
Me-Ala 0.04 Phe
Me2-Ala 8.7
>20 Leu
0.16 Me-Ala 1.6
5
Nle
Lys
0.12 Hyp
0.05 Pip
0.12 Aze
0.04
0.8
0.4
1.8
0.12 Tyr
Cl-Phe
Tle
D-Ala
D-Val
D-Phe
Aib
>20
1-Nal
2-Nal
Phg
homo-Phe 0.18
D-Phe
Aib
>20
>20
>20
Me-Ala >20
0.8
6.7
a
All analogues were derived from the pentapeptide AVPFY.
Four libraries were synthesized in which each of the first four
residues was varied while keeping the other four residues un-
changed. Abbreviations: R-aminobutyric acid (Abu), R-aminoisobu-
tyric acid (Aib), azetidine-2-carboxylic acid (Aze), cyclohexylalanine
(Cha), 4-chlorophenylalanine (Cl-Phe), (4R)-4-hydroxyproline (Hyp),
1-naphthylalanine (1-Nal), 2-naphthylalanine (2-Nal), norleucine
(Nle), phenylglycine (Phg), pipecolic acid (Pip), sarcosine (Sar), tert-
leucine (Tle). Unless indicated otherwise, amino acids have the
L-configuration.
the therapeutic application of peptides is hampered in
many cases by their proteolytic instability, limited
cellular penetration, and poor pharmacokinetics.
Our goal was to improve the potency and proteolytic
stability of the SMAC peptides to provide a compound
that could be used to evaluate XIAP as a target for
cancer chemotherapy in vitro and in vivo. In the present
study, we report on the discovery of a series of pro-
teolytically stable, capped tripeptides comprised of
unnatural amino acids that bind to the BIR3 domain of
XIAP with single-digit nanomolar affinity, display cy-
totoxic effects in cancer cell lines, and slow the growth
of tumors in a MDA-MB-231 breast cancer mouse
xenograft model.
Resu lts a n d Discu ssion
SAR fr om P ep tid e Libr a r ies. To rapidly determine
the structure/activity relationships (SAR) for the bind-
ing of the individual residues of the SMAC N-terminal
tetrapeptide, peptide libraries were synthesized employ-
ing high-throughput solid-phase peptide synthesis. We
have previously shown that the N-terminus of processed
HID, a functional homologue of SMAC found in lower
organisms,18 has a 4-fold higher binding affinity for the
BIR3 domain of XIAP when compared to the corre-
sponding SMAC peptide.11 Therefore, as a starting
point, it was decided to use AVPFY, the pentapeptide
N-terminus of processed HID, that binds to the XIAP-
BIR3 domain with an affinity of 60 nM. Four libraries
were prepared in which each of the first four residues
was varied while keeping the other four residues
unchanged. A total of more than 120 compounds was
synthesized and evaluated for binding to the BIR3
domain of XIAP by employing a fluorescent polarization
assay.11 Table 1 provides a summary of the BIR3
binding affinities for a representative group of peptides
from these libraries. For position 1, only small residues
with the natural L-configuration (e.g., Ala or Abu) are
SAR fr om Ca p p ed Tr ip ep tid e Libr a r ies. To re-
duce the peptide character of the pentapeptide lead and
improve the binding affinity, a series of more than 150
capped tripeptides was designed and synthesized in a
high-throughput parallel format (Scheme 1). Assembly

Antagonists of the Antiapoptotic Protein XIAP
J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18 4419
enclosed in the Supporting Information. Analogues
bearing phenyl-alkyl amide caps with linear alkyl
spacers and optional ring substitutions showed only
moderate to weak binding to the BIR3 protein. However,
binding increased to e50 nM levels with benzyl amide
caps that are branched in the benzylic position, with
the tetrahydronaphthyl amide being the most active cap
in the first round. From the SAR of the peptide libraries
(Table 1), it was concluded that Ala in position 1 and
Val in position 2 can be replaced with N-Me-Ala and
Tle (tert-leucine), respectively, without loss of binding
affinity. Combination of these unnatural amino acid
residues with the most promising amide caps discovered
in the first library led to the development of a second
library. The binding affinities of representative ex-
amples from this library are presented in Table 2.
Conformational restriction of the aromatic ring in the
(R)-tetrahydronaphthyl amide analogue 2 and (R)-
indane amide analogue 4 resulted in potent ligands of
XIAP-BIR3 (Kd ) 0.012 and 0.029 µM, respectively).
The diastereomer analogues 3 and 5 showed markedly
reduced binding affinities, suggesting that only the (R)-
configuration places the aromatic ring in a favorable
position for binding. A model of compound 2 docked into
the BIR3 domain of XIAP indicated that the (R)-
configuration of the tetrahydronaphthyl amide cap
corresponds to the L-configuration of position 4 amino
acids. Thus, the (R)-tetrahydronaphthyl and indane
amide groups can be regarded as conformationally
restricted versions of L-Phg, which is the amino acid that
afforded the best binding affinity in this position (Table
1). The structure of compound 2 bound to XIAP-BIR3
was determined and is shown in Figure 1B. The
structure confirms that only the (R)-configuration pro-
vides a vector that places the tetrahydronaphthyl group
in the correct geometry for interaction with the hydro-
phobic alkyl portions of Lys 297 and Lys 299. This
substitution causes the proline at position 3 and the
phenyl ring of the tetrahydronaphthyl at position 4 to
fill more of the pocket than the corresponding Pro and
Ile residues of the original SMAC tetrapeptide (Figure
1A). This change is supported by NOE contacts between
Leu 292 and the tetrahydronaphthyl at position 4, which
were not observed for the Ile at position 4 in the SMAC
tetrapeptide complex.11 Attachment of achiral 1- and
2-naphthyl amide caps (sp2 connection) resulted in the
inactive analogues 6 and 7. In contrast, the less planar
fluorenyl amide analogue 8 (sp3 connection) was well-
tolerated, underscoring the subtle conformational pref-
erences for this position. Overall, optimization of the cap
portion resulted in compounds 2 and 4 with low double-
digit nanomolar binding affinity for the XIAP-BIR3
domain. Importantly, compound 2 was shown to be
stable in plasma and mouse whole blood (2 h incubation
at 37 °C) and during the time needed to conduct the
cell-based experiments (data not shown).
F igu r e 1. (A) A transparent solvent-accessible surface show-
ing the AVPI terminus of the SMAC peptide bound to the BIR3
domain of XIAP11 (PDB ID: 1G3F). Two potential sites for
improving the binding affinity of the tetrapeptide lead are
indicated by red ovals. (B) Solvent-accessible surface showing
compound 2 bound to the BIR3 domain of XIAP (PDB ID:
1TFQ). (C) Compound 10 bound to the BIR3 domain of XIAP
(PDB ID: 1TFT). Leu, Ile, Val, Met, Tyr, Phe, Trp, and the
hydrophobic side chain of Lys are colored yellow. Asp and Glu
are colored red. Arg, His and the amino group of Lys are
colored blue. All other residue types are colored gray. The side
chains for Lys 297, Lys 299, Trp 323, and Tyr 324 are shown.
of the benzyl protected tripeptide Boc-AVP-OBn was
conducted using standard peptide coupling conditions.
After hydrogenolytic cleavage of the benzyl ester, the
resulting acid Boc-AVP-OH was coupled to a variety of
amines employing solid-phase reagents. Epimerization
is of no concern in this coupling reaction, since the
carboxylic acid function is part of a proline residue, and,
thus, oxazolone formation is impossible. Given the
observed preference for aromatic residues in position 4,
mostly amines comprising an aromatic or heteroaro-
matic ring were selected for C-terminal capping. The
final products were obtained after removal of the Boc
protecting group and preparative HPLC purification.
The BIR3 binding data observed for this first library is
To further enhance the binding affinity of the pepti-
domimetic BIR3 ligands, additional sites not accessed
by the original peptides were pursued. From an analysis
of the structure of the SMAC/XIAP-BIR3 complex it
appeared that an additional hydrophobic groove made
up by W323 and Y324 (Figure 1A) could be accessed by
modification of the 4-position of the Pro residue. Par-
ticularly, the tetrahydronaphthyl cap in compound 2

4420 J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18
Sch em e 1. Synthesis of Capped Tripeptide Librariesa
Oost et al.
a
Reagents and conditions: (a) Boc-Val-OH or Boc-Tle-OH, EDCI, HOBt, DIPEA, DMF; (b) 4 N HCl in dioxane; (c) Boc-Ala-OH or
Boc-N-Me-Ala-OH, EDCI, HOBt, DIPEA, DMF; (d) H2, Pd on carbon, MeOH; (e) Cap-NH2, PS-Carbodiimide resin, HOAt, PS-Trisamine
resin, DMA/DCE 1:1; (f) TFA/H2O/TIS (95:2.5:2.5).
Ta ble 2. Binding Affinity to the XIAP-BIR3 Domain for
Representative Examples from the Second Library of Capped
Tripeptides
logues 2 and 11 (Table 3). A structure of compound 10
complexed to XIAP-BIR3 is shown in Figure 1C. This
compound binds to BIR3 in a manner similar to com-
pound 2 (Figure 1B) and the SMAC peptide (Figure 1A).
The phenoxy substituent at the Pro 4-position leans
against Trp 323 of BIR3, and stacking of the aromatic
rings may contribute favorably to compound binding.
Compounds 10 and 12 that are comprised solely of
unnatural amino acid residues afforded single-digit
nanomolar binding affinity for the BIR3 domain of
XIAP.
P r oa p op totic Activity of XIAP -BIR3 Liga n d s in
Recon stitu ted Assa ys. To evaluate the ability of the
BIR3 ligands to mimic the proapoptotic protein SMAC
by relieving the inhibition of caspases by XIAP, selected
analogues were tested in an in vitro assay system
described previously.21 In the fully reconstituted assay
containing Apaf-1, caspase-9 (p35/p12), procaspase-3,
dATP and cytochrome c, the potent BIR3 ligands 2, 10,
and 11 (Kd ) 0.005-0.016 µM) were found to rescue
BIR3-mediated inhibition of caspase activity at submi-
cromolar levels (Table 4). Compound 9 (phenethyl amide
cap) as well as full-length SMAC afforded micromolar
EC50 values, which is in line with their diminished BIR3
binding affinity (Kd ) 0.17 and 0.42 µM, respectively).
However, in a reconstituted assay inhibited by full-
length XIAP instead of BIR3, the activity of recombinant
SMAC increased dramatically, by more than 2 orders
of magnitude; in contrast, the activity of the BIR3
ligands 2, 10, and 11 was virtually unchanged. These
findings suggest that the SMAC-XIAP interaction is
not solely mediated through the BIR3 interface, and give
support to a model in which SMAC interacts with XIAP
as a dimer, likely involving interactions with other
domains of XIAP, such as BIR2. Not surprisingly, these
additional binding interactions cannot be mimicked by
small molecules that were optimized solely for binding
to the BIR3 domain of XIAP. Our findings are consistent
with recent results which show that simultaneous
interaction with both BIR2 and BIR3 domains are
required for the relief of XIAP-mediated caspase inhibi-
tion by SMAC and suggest a 2:1 SMAC/XIAP interaction
stoichiometry.22 The N,N-dimethyl-Ala analogue 14,
which does not bind to the XIAP-BIR3 domain (Table
3), was inactive in the reconstituted assays. Thus,
within the capped tripeptide series, the results of the
reconstituted assay correlate with the direct measure-
ment of binding to the BIR3 domain.
a
Values are representative of at least two measurements.
causes a slight change in the Pro position and improves
the vector from the Pro 4-position into this pocket
(Figure 1B). Several capped tripeptides that bear hy-
drophobic substituents in the 4-position of Pro were
prepared as outlined in Scheme 2 by employing stan-
dard peptide coupling protocols. In fact, compounds 10
and 12 comprising a (4S)-4-phenoxy-Pro residue in
position 3 (tert-leucine and cyclohexylglycine in position
2, respectively) showed an almost 3-fold improved
binding affinity compared to the unsubstituted ana-

Antagonists of the Antiapoptotic Protein XIAP
J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18 4421
Sch em e 2. Synthesis of Prosubstituted Capped Tripeptidesa
a
Reagents and conditions: (a) Boc-Pro-OH or (4S)-Boc-4-phenoxy-Pro-OH, EDCI, HOBt, DIPEA, DMF; (b) 4 N HCl in dioxane; (c)
Boc-Tle-OH or Boc-Chg-OH, EDCI, HOBt, DIPEA, DMF; (d) Boc-N-Me-Ala-OH or N,N-Me2-Ala-OH or Boc-N-Me-D-Ala-OH, EDCI, HOBt,
DIPEA, DMF. (4S)-Boc-4-phenoxy-Pro-OH was synthesized from commercially available Boc-Hyp-OMe in two steps.20
Ta ble 3. Binding Affinity to the XIAP-BIR3 Domain of Pro-
acute promyelocytic leukemia HL-60. There was no
correlation between potency of compounds 2 and 11 and
expression of XIAP,2 suggesting that other factors
contribute to sensitivity of the various cell lines. A
sample dose-response curve of killing of MDA-MB-231
cells is shown in Figure 2A. The toxicity exhibited by
the peptidomimetic BIR3 ligands in the sensitive cell
lines occurs in the absence of an identified apoptotic
stimulus. This observation is distinct from previous
reports in which peptides with lower binding affinity
for XIAP potentiated the toxicity of chemotherapeutic
agents14-16 or death receptor agonists.15,17 To examine
if BIR3 binding correlated with toxicity in cancer cell
lines, negative control compounds with close structural
similarity to the active analogues 2 and 11 were
synthesized based on the SAR derived from the peptide
libraries (Tables 1 and 2). Negative control compound
15 contains a N,N-dimethyl-Ala terminus, compound 16
is comprised of a N-methyl-D-Ala terminus, and com-
pound 3 bears a mismatch C-terminal cap ((S)-tetrahy-
dronaphthyl). In all cases, the weak BIR3 binding of
these negative controls (Table 3) correlated well with
strongly diminished cellular toxicity, consistent with the
hypothesis that the active analogues kill susceptible
human cancer cell lines in a mechanism-specific man-
ner.
and Ala-Modified Capped Tripeptides
Ala
BIR3
compound
R1
R2
R3 configuration Kd[µM] a
2
10
11
12
13
14
15
16
H
tert-butyl
H
H
H
H
L
L
L
L
L
L
L
D
0.012
0.005
0.016
0.006
>1
phenoxy tert-butyl
cyclohexyl
phenoxy cyclohexyl
H
H
tert-butyl Me
phenoxy tert-butyl Me
H
H
>1
cyclohexyl Me
tert-butyl
>1
>1
H
a
Values are representative of at least two measurements.
Ta ble 4. Relief of Caspase Inhibition by BIR3 Ligands in
Reconstituted Assaysa
BIR3 + APAF-1 +
XIAP + APAF-1 +
casp9 + procasp3 procasp9 + procasp3
BIR3
Kd [µM]
compound
EC50 [µM]b
EC50 [µM]b
2
10
11
14
0.29
0.24
0.31
>100
1.26
2.08
0.49
0.71
0.47
>100
4.21
0.01
0.012
0.005
0.016
>1
0.167
0.42
Inhibition of IAPs by SMAC induces caspase-depend-
ent apoptosis.9 Thus, it was predicted that the BIR3
ligands would induce caspase-3 activity in cells, and that
caspase inhibition would promote cell survival. The
susceptible mammary gland adenocarcinoma MDA-MB-
231 cell line was exposed to either 1 µM or 10 nM
concentrations of compound 11 for up to 30 h and
caspase-3 activity was measured in cell lysates using
DEVD-amc as a substrate. An up to 8-fold increase in
caspase-3 activity was observed in a dose-dependent
manner at 12-18 h following compound exposure
(Figure 3). Exposure to 1 µM of the negative control 15
(N,N-dimethyl-Ala analogue) resulted in caspase-3 ac-
tivity comparable to that of the suboptimal killing
concentration of 10 nM of compound 11. Incubation of
compound 11-treated cells with 10 µM of the pan-
caspase inhibitor IDN-537023 reduced caspase-3 activity
to the background level seen in untreated cells (Figure
3). Caspase inhibitor treatment completely protected
MDA-MB-231 cells from an otherwise lethal concentra-
tion of compound 11 (Figure 2B) with an EC50 of 1 µM,
9
SMAC
a
Activity is expressed as an EC50, the concentration at which
b
half-maximum recovery was achieved. Values are representative
of at least two observations.
Cytotoxic Activity of XIAP -BIR3 Liga n d s in
Hu m a n Ca n cer Cell Lin es. To determine whether the
proapoptotic activity that was observed in the reconsti-
tuted assays translates into cellular activity, the cyto-
toxicity of selected BIR3 ligands was assessed against
various cancer cell lines (Table 5). Compounds 2 and
11 show a wide range of potencies against a panel of 59
human cancer cell lines ranging from single-digit na-
nomolar activity in some cell lines to no activity at 50
µM in most others. Both compounds exhibited similar
activities in the cell lines in which they were examined,
with compound 11 being slightly more potent than
compound 2. The highest potency was observed against
the human breast cancer cell lines BT-549 and MDA-
MB-231, the melanoma cell line SK-MEL-5, and the

4422 J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18
Oost et al.
Ta ble 5. Toxicity of BIR3 Ligands against the Eight Most Sensitive Human Cancer Cell Linesa
exposure
cell line
BT-549
MDA-MB-231 breast
HL-60
SK-MEL-5
RXF-393
SK-OV-3
NCI-H23
NCI-H522
tissue type
breast
time (h)
2
3
16
11
15
72
48
72
48
72
48
72
72
0.015 ( 0.011 (4)
n.a.b
n.a.
0.007 ( 0.00035 (2) 1 ( 0.47 (4)
0.068 ( 0.031 (10) 3 ( 0.14 (2)
8.3 ( 1.8 (2) 0.013 ( 0.0056 (10) 3.4 ( 1.1 (8)
leukemia
melanoma
renal
ovarian
NSCL
NSCL
0.99 ( 1 (2)
n.a.
n.a.
n.a.
n.a.
n.a.
n.a.
0.19 ( 0.13 (2)
0.29 ( 0.22 (4)
1.1 ( 0.14 (2)
1.3 ( 1.1 (6)
1.5 ( 0.46 (4)
2 ( 0.64 (2)
4.3 ( 3.1 (2)
15 ( 6.6 (4)
>50 (2)
1.3 ( 0.86 (4)
0.81 ( 0.25 (2)
1.5 ( 1.2 (6)
5.3 ( 2.1 (4)
1.7 ( 0.14 (2)
5.6 ( 0.42 (2) >50 (2)
>25 (4)
n.a.
n.a.
n.a.
n.a.
>50 (4)
>50 (2)
a
An additional 52 cell lines tested showed less sensitivity to compound 11 and are listed in the Supporting Information. Compounds
3 and 16 are closely related in structure to compound 2, but substantially less potent BIR3 ligands (Tables 2 and 3). Similarily, compound
15 is closely related to compound 11. Cell lines were exposed to the compounds for 48 or 72 h in the presence of 10% FBS and viability
b
was measured by MTS reduction. Mean EC50 [µM] ( SD (n). n.a., not available.
F igu r e 3. Time course of caspase-3 activity in cell lysates of
MDA-MB-231 cells exposed to compounds 11 and 15 and the
broad-spectrum caspase inhibitor IDN-5370.Maximum caspase-3
activity was observed at 12-18 h after addition of 10 nM to 1
µM of the BIR3 potent ligand 11. Exposure of the cells to 10
µM of IDN-5370 during the time of incubation with 1 µM of
compound 11 inhibited caspase-3 quantitatively.
F igu r e 2. (A) Dose response curves of killing of MDA-MB-
231 cells by compound 11. MDA-MB-231 cells were cultured
in 96-well plates for 48 h in the presence of various concentra-
tions of compound 11 and viability was measured. (B) Promo-
tion of survival of compound 11-exposed MDA-MB-231 cells
by the pan-caspase inhibitor IDN-5370. MDA-MB-231 cells
were exposed to 10 µM of compound 11 and variable concen-
trations of the pan-caspase inhibitors IDN-5370 for 48 h,
F igu r e 4. Comparison of efficacy of compound 2 (40 mg/kg/
day, sc), compound 11 (20 mg/kg/day, sc), and negative control
13 (40 mg/kg/day, sc) to paclitaxel given ip at the MTD of 30
mg/kg/day. Both compound 2 and compound 11 showed a
followed by measuring viability.
highly significant inhibition of tumor growth throughout the
study. The efficacy of the BIR3 ligands was similar to pacli-
taxel during the dosing period, but tumor growth resumed
after cessation of therapy. A repeat study with compounds 2
and 11 showed similar results.
which is consistent with the potency of this caspase
inhibitor in other cellular models of apoptosis.23 Taken
together, these results provide further evidence for
mechanism-specific cell killing by active BIR3 ligands
in susceptible human cancer cell lines.
(EC50 g 28 µM) was also analyzed. The BIR3 ligands 2
and 11 showed significant efficacy in the MDA-MB-231
model (Figure 4). Consistent with in vitro analyses,
compound 11 appeared slightly more potent in vivo than
compound 2. Compound 2 given at 40 mg/kg/day and
compound 11 given at 20 mg/kg/day (near their maxi-
mum tolerated dose based on skin toxicity) each inhib-
ited the tumor growth rate by 50-60% relative to the
vehicle control. During the dosing period (day 14-30)
tumor growth was essentially flat, and efficacy was
In Vivo Effica cy of XIAP -BIR3 Liga n d s in th e
MDA-MB-231 F la n k Br ea st Ca n cer Xen ogr a ft
Mod el. We sought to investigate the efficacy of the
potent BIR3 ligands 2 and 11 in the MDA-MB-231 flank
breast cancer xenograft model. This model was chosen
due to the remarkable in vitro sensitivity of this cell
line to these compounds (EC50 ) 0.068 and 0.013 µM
for 2 and 11, respectively, Table 5). As a negative
control, the inactive N,N-dimethyl-Ala analogue 13

Antagonists of the Antiapoptotic Protein XIAP
J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18 4423
M solution of HOAt (500 µmol) in DMF. N,N′-diisopropylcar-
bodiimide (79 µL, 500 µmol each) was added, and the solutions
were allowed to stand for 5 min before coupling. If a position
of the pentapeptide was not varied, a stock solution (100 mL)
of the respective amino acid (Ala, Val, or Pro) was prepared.
comparable to that observed for paclitaxel. However,
unlike with paclitaxel treatment, tumor growth after
termination of therapy resumed at a rate comparable
to the vehicle control. Although the negative control
compound 13 did show a modest separation from the
vehicle control, the effect was transient and significantly
less robust than that observed for the potent BIR3
ligands.
Coupling: Solutions of the activated amino acids (2 mL)
were added to the respective chambers in the Robbins block
and the block was agitated for 1 h. After filtration, the resins
were washed three times with 4 mL of DMF. The deprotection/
coupling cycle was repeated until the pentapeptide stage. Each
resin was washed twice with 4 mL of DMF, MeOH, and diethyl
ether. To cleave the pentapeptides and remove the side chain
protecting groups, 1 mL of a TFA/H2O/TIS cocktail (95:2.5:
2.5) was added to each chamber containing the air-dry resins
and the block was agitated for 1 h. The peptide solutions were
filtered into a Robbins collection block and the resins were
washed twice with TFA. After drying the sample in a Genevac
HT4 evaporator, the remaining residues were dissolved in
water/MeOH (1:1) and purified by preparative RP-HPLC. The
pentapeptides were isolated as TFA salts in >95% purity (LC/
MS and 1H NMR analysis).
Gen er a l P r oced u r e for Solu tion -P h a se P ep tid e Cou p -
lin g. EDCI (6.33 g, 33.0 mmol) was added to a stirred solution
of the Boc-protected amino acid (30.0 mmol), DIPEA (5.23 mL,
30.0 mmol), and HOBt (4.46 g, 33.0 mmol) in DMF (50 mL) at
0 °C. If the amine coupling partner was used as the HCl salt,
the amount of DIPEA was doubled. After 5 min, the amino
component (33.0 mmol) was added to the ice-cold coupling
solution, and the reaction was stirred overnight at room temp.
The mixture was diluted with EtOAc and washed twice with
1 N aqueous HCl, NaHCO3 solution and brine, respectively.
The organic layer was dried over MgSO4, and the volatiles
were removed under reduced pressure. The coupling product
was purified by flash chromatography (gradient of EtOAc in
hexane).
Gen er a l P r oced u r e for Rem ova l of th e Boc P r otectin g
Gr ou p . The Boc-protected compound (30 mmol) was stirred
with 4 N HCl in dioxane (50 mL) for 1 h at room temp. The
volatiles were removed under reduced pressure, and the
residue was coevaporated twice with diethyl ether. The HCl
salt was dried in vacuo.
Syn th esis of Ca p p ed Tr ip ep tid e Libr a r y. Boc-N-Me-
Ala -Tle-P r o-OH (1). Utilizing the general procedures for
peptide coupling and Boc removal, the benzyl-protected tri-
peptide Boc-N-Me-Ala-Tle-Pro-OBn was synthesized (H-Pro-
OBn HCl salt, Boc-Tle-OH, Boc-N-Me-Ala as the coupling
components). To a solution of the benzyl ester (2.35 g, 4.67
mmol) in MeOH (50 mL) was added palladium on charcoal
(5%, 100 mg), and the reaction was stirred under a hydrogen
atmosphere for 18 h. The catalyst was filtered off over Celite,
and the filtrate was concentrated under reduced pressure to
yield 1.76 g of a white foam (92%). 1H NMR (500 MHz, MeOH-
d4) δ 4.61 (m, 2H), 4.41 (dd, 1H), 3.87 (m, 1H), 3.73 (m, 1H),
2.85 (s, 3H), 2.27 (m, 1H), 1.95-2.10 (m, 3H), 1.47 (s, 9H), 1.33
(d, 3H), 1.05 (s, 9H). MS m/z ) 414 (M+H)+, 436 (M+Na)+.
Con clu sion
Starting from a micromolar natural peptide lead, a
series of capped tripeptides was designed that contain
unnatural amino acids and bind to the BIR3 domain of
XIAP with single-digit nanomolar affinity. The proapo-
ptotic activity of these compounds was demonstrated in
an in vitro system that reconstitutes the cell’s apoptotic
machinery. Furthermore, these compounds exhibited
remarkable cytotoxic activity in several cancer cell lines.
Cytotoxic activity in the MDA-MB-231 breast cancer cell
line translated into efficacy in the corresponding mouse
xenograft model. The present study suggests that the
BIR3 domain of XIAP represents a promising target for
the development of small molecules that antagonize
XIAP function and induce apoptosis.
Exp er im en ta l Section
Gen er a l. 1H NMR spectra were obtained at 300 or 500 MHz
using tetramethylsilane as the internal standard. For all
proline-containing analogues two sets of signals originating
from the two conformers were observed (ratio 9:1 to 3:1); the
signal set for the major conformer is reported. Mass spectra
(electron spray ionization mode) were recorded on Finnigan
API SSQ7000 instruments. High-resolution mass spectra (fast
atom bombardment mode) were recorded on a J EOL SX102A
instrument. Flash chromatography was carried out on RediSep
silicagel cartridges (ISCO) using an ISCO SQ100c chromatog-
raphy station. Reactions were routinely conducted under
nitrogen using commercial high purity solvents as received.
Amino acid derivatives were purchased from Bachem, Fluka
or NovaBiochem. Unless indicated otherwise, amino acids have
the L-configuration. (4S)-Boc-4-phenoxy-Pro-OH was synthe-
sized according to a literature reference.20 LC/MS analysis was
performed with a Phenomenex Luna C8 column (2.1 × 30 mm).
TFA+ method: Gradient from 10 to 100% acetonitrile in 0.1%
aqueous TFA (over 3.1 min; flow rate ) 1.5 mL/min); AA+
method: As before, except 10 mM aqueous ammonium acetate
instead of 0.1% aqueous TFA; MS data were aquired with
APCI, positive ionization; UV detection at 220 nM. Preparative
reversed phase HPLC was performed on a Waters Nova-Pak
HR C18 6µm 60Å column (25 × 100 mm) using a gradient of
10 to 100% acetonitrile in 0.1% aqueous TFA (over 8 min; flow
rate ) 40 mL/min). Abbreviations: Cyclohexyl-glycine (Chg),
1,2-dichloroethane (DCE), N,N-dimethylacetamide (DMA),
N,N-diisopropylethylamine (DIPEA), 1-ethyl-3-(3-dimethy-
laminopropyl)carbodiimide (EDCI), 1-hydroxy-7-azabenzotria-
zole (HOAt), 1-hydroxybenzotriazole (HOBt), trifluoroacetic
acid (TFA), triisopropylsilane (TIS), tert-leucine/tert-butyl-
glycine (Tle).
Cou p lin g of Tr ip ep tid e Acid 1 w ith Am in o Ca p s. The
chambers of a Robbins reaction block were filled with PS-
Carbodiimide resin (Argonaut Technologies, 70 mg, f ) 2
mmol/g, 140 µmol). To each chamber was added a solution of
the respective amine (70 µmol, 0.35 mL of a 0.2 M solution in
DMA), a solution of the tripeptide acid 1 (85 µmol, 0.85 mL of
a 0.1 M solution in 20% DMA in DCE), and a solution of HOAt
(120 µmol, 0.6 mL of 0.2 M solution in 20% DMA in DCE).
The reaction block was shaken for 18 h at room temp. To
scavenge excess acid, PS-Trisamine resin (Argonaut Technolo-
gies, 70 mg, f ) 3 mmol/g, 210 µmol) was added to each
reaction chamber, and the block was agitated for an additional
3 h. Then, the reaction solutions were filtered into a Robbins
collection block and the volatiles were removed in a Genevac
HT4 evaporator. To remove the N-terminal Boc-group, the
residue in each chamber was treated with 1 mL of a TFA/H2O/
TIS cocktail (95:2.5:2.5) for 1 h. After drying the sample in
the evaporator, the remaing residues were dissolved in water/
MeOH (1:1) and purified by preparative RP-HPLC. The capped
P a r a llel Syn th esis of P ep tid e Libr a r ies. The chambers
of a Robbins reaction block (Robbins Scientific) were filled with
50 µmol of Rink MBHA resin (Novabiochem) carrying the
following amino acid sequences: Position 1 library: Fmoc-Val-
Pro-Phe-Tyr(tBu)-NH-resin; position 2 library: Fmoc-Pro-Phe-
Tyr(tBu)-NH-resin; position 3 library: Fmoc-Phe-Tyr(tBu)-NH-
resin; position 4 library: Fmoc-Tyr(tBu)-NH-resin.
Removal of Fmoc group: The resins in each chamber were
treated twice with 4 mL of 20% piperidine in DMF (3 min each
cycle). After filtration, all resins were washed three times with
4 mL of DMF. Amino acid activation: The Fmoc-protected
amino acids (500 µmol) were each dissolved in 2 mL of a 0.25

4424 J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18
Oost et al.
Ta ble 6
H -N -Me -Ala -T le -(4S )-4-p h e n o x y -P r o -(R )-t e t r a h y -
d r on a p h th -1-yl Am id e (10). tR (TFA+) ) 1.78 min; tR (AA+)
) 1.93 min. 1H NMR (500 MHz, D2O) δ 7.05-7.35 (m, 6H),
6.67-7.00 (m, 3H), 5.11 (m, 1H), 4.93 (m, 1H), 4.57 (m, 1H),
4.44 (s, 1H), 3.83-4.05 (m, 3H), 2.52-2.75 (m, 6H), 2.38 (m,
1H), 1.55-1.85 (m, 3H), 1.39-1.50 (m, 4H), 0.87 (s, 9H). MS
m/z ) 535 (M+H)+. HRMS for [C31H43N4O4]+ 535.3284; found
535.3279.
H-N-Me-Ala-Ch g-P r o-(R)-tetr ah ydr on aph th -1-yl Am ide
(11). tR (TFA+) ) 1.58 min; tR (AA+) ) 1.66 min. 1H NMR
(500 MHz, D2O) δ 7.13-7.25 (m, 4H), 4.96 (m, 1H), 4.55 (d,
1H), 4.35 (m, 1H), 3.98 (m, 1H), 3.87 (m, 1H), 3.73 (m, 1H),
2.77 (m, 2H), 2.70 (s, 3H), 2.30 (m, 1H), 1.93-2.13 (m, 4H),
1.57-1.90 (m, 9H), 1.50 (d, 3H), 1.05-1.30 (m, 5H). MS m/z )
469 (M+H)+. HRMS for [C27H41N4O3]+ 469.3179; found
469.3185.
H -N -Me -Ala -C h g -(4S )-4-p h e n o x y -P r o -(R )-t e t r a h y -
d r on a p h th -1-yl Am id e (12). tR (TFA+) ) 1.83 min; tR (AA+)
) 2.05 min. 1H NMR (500 MHz, D2O) δ 6.53-7.30 (m, 9H),
5.09 (m, 1H), 4.92 (m, 1H), 4.51 (d, 1H), 4.30 (d, 1H), 3.84-
3.98 (m, 3H), 2.54-2.73 (m, 6H), 2.38 (d, 1H), 1.33-1.87 (m,
13H), 0.72-1.02 (m, 5H). MS m/z ) 561 (M+H)+. HRMS for
[C33H45N4O4]+ 561.3441; found 561.3441.
H-N,N-Me2-Ala-Tle-P ro-(R)-tetrahydronaphth-1-ylAmide
(13). tR (TFA+) ) 1.46 min; tR (AA+) ) 1.85 min. 1H NMR
(500 MHz, D2O) δ 7.19-7.34 (m, 4H), 4.99 (t, 1H), 4.64 (s, 1H),
4.40 (m, 1H), 4.10 (m, 1H), 3.95 (m, 1H), 3.76 (m, 1H), 2.92 (s,
6H), 2.80 (m, 2H), 2.31 (m, 1H), 2.12 (m, 1H), 1.73-2.05 (m,
6H), 1.54 (d, 3H), 1.10 (s, 9H). MS m/z ) 457 (M+H)+. HRMS
for [C26H41N4O3]+ 457.3179; found 457.3182.
second
amino acid
compd
first amino acid
third amino acid
2
10
Boc-N-Me-Ala-OH
Boc-N-Me-Ala-OH
Boc-Tle-OH Boc-Pro-OH
Boc-Tle-OH (4S)-Boc-4-phenoxy-
Pro-OH
Boc-Chg-OH Boc-Pro-OH
Boc-Chg-OH (4S)-Boc-4-phenoxy-
Pro-OH
Boc-Tle-OH Boc-Pro-OH
Boc-Tle-OH (4S)-Boc-4-phenoxy-
Pro-OH
11
12
Boc-N-Me-Ala-OH
Boc-N-Me-Ala-OH
13
14
N,N-Me2-Ala-OH
N,N-Me2-Ala-OH
15
16
N,N-Me2-Ala-OH
Boc-Chg-OH Boc-Pro-OH
Boc-N-Me-D-Ala-OH Boc-Tle-OH Boc-Pro-OH
tripeptides were isolated as the TFA salts in 50-80% yield
and >95% purity (LC/MS analysis).
H-N-Me-Ala -Tle-P r o-(R)-tetr a h yd r on a p h th -1-yl Am id e
(2). tR (TFA+) ) 1.47 min; tR (AA+) ) 1.55 min. 1H NMR (500
MHz, D2O) δ 7.19-7.35 (m, 4H), 4.99 (m, 1H), 4.64 (s, 1H),
4.39 (m, 1H), 4.07 (m, 1H), 3.95 (m, 1H), 3.77 (m, 1H), 2.80
(m, 2H), 2.71 (s, 3H), 2.30 (m, 1H), 1.77-2.17 (m, 7H), 1.50
(d, 3H), 1.10 (s, 9H). MS m/z ) 443 (M+H)+. HRMS for
[C25H39N4O3]+ 443.3022; found 443.3015.
H-N-Me-Ala -Tle-P r o-(S)-tetr a h yd r on a p h th -1-yl Am id e
(3). tR (TFA+) ) 1.49 min; tR (AA+) ) 1.56 min. 1H NMR (500
MHz, MeOH-d4) δ 7.05-7.17 (m, 4H), 5.02 (t, 1H), 4.63 (s, 1H),
4.38 (m, 1H), 3.93 (m, 2H), 3.74 (m, 1H), 2.70-2.85 (m, 2H),
2.66 (s, 3H), 1.75-2.23 (m, 8H), 1.45 (d, 3H), 1.10 (s, 9H). MS
m/z ) 443 (M+H)+.
H-N-Me-Ala-Tle-P r o-(R)-in dan -1-yl Am ide (4). tR (TFA+)
) 1.41 min; tR (AA+) ) 1.47 min. H NMR (500 MHz, MeOH-
d4) δ 7.40 (d, 1H), 7.10-7.25 (m, 3H), 5.35 (t, 1H), 4.66 (s, 1H),
4.38 (m, 1H), 3.93 (m, 2H), 3.75 (m, 1H), 2.98 (m, 1H), 2.87
(m, 1H), 2.66 (s, 3H), 2.47 (m, 1H), 2.10-2.25 (m, 2H), 1.80-
2.02 (m, 3H), 1.46 (d, 3H), 1.13 (s, 9H). MS m/z )429 (M+H)+.
HRMS for [C24H37N4O3]+ 429.2866; found 429.2868.
H -N,N -Me 2-Ala -Tle-(4S)-4-p h en oxy-P r o-(R)-t et r a h y-
d r on a p h th -1-yl Am id e (14). tR (TFA+) ) 1.80 min; tR (AA+)
1
1
) 2.23 min. H NMR (300 MHz, DMSO-d6) δ 9.75 (br s, 1H),
8.76 (d, 1H), 8.11 (d, 1H), 6.82-7.35 (m, 10H), 5.06 (m, 1H),
4.95 (m, 1H), 4.48 (m, 2H), 4.23 (m, 1H), 4.05 (m, 1H), 3.68
(m, 1H), 2.52-2.82 (m, 9H), 1.55-2.10 (m, 4H), 1.37 (d, 3H),
1.02 (s, 9H). MS m/z ) 549 (M+H)+. HRMS for [C32H45N4O4]+
549.3441; found 549.3455.
H-N-Me-Ala -Tle-P r o-(S)-in d a n -1-yl Am id e (5). tR (TFA+)
) 1.42 min; tR (AA+) ) 1.47 min. H NMR (500 MHz, MeOH-
H-N,N-Me2-Ala-Chg-P ro-(R)-tetrahydronaphth-1-ylAmide
(15). tR (TFA+) ) 1.57 min; tR (AA+) ) 1.96 min. 1H NMR
(500 MHz, D2O) δ 7.19-7.31 (m, 4H), 4.99 (t, 1H), 4.57 (d, 1H),
4.39 (m, 1H), 4.04 (m, 1H), 3.91 (m, 1H), 3.75 (m, 1H), 2.91 (s,
6H), 2.80 (m, 2H), 2.31 (m, 1H), 1.62-2.15 (m, 13H), 1.54 (d,
3H), 1.05-1.30 (m, 5H). MS m/z ) 483 (M+H)+. HRMS for
[C28H43N4O3]+ 483.3335; found 483.3350.
1
d4) δ 7.20 (m, 4H), 5.33 (t, 1H), 4.64 (s, 1H), 4.40 (m, 1H), 3.93
(m, 2H), 3.75 (m, 1H), 3.00 (m, 1H), 2.86 (m, 1H), 2.66 (s, 3H),
2.50 (m, 1H), 1.87-2.25 (m, 5H), 1.46 (d, 3H), 1.10 (s, 9H).
MS m/z ) 429 (M+H)+.
H-N-Me-Ala -Tle-P r o-n a p h th -1-yl Am id e (6). tR (TFA+)
) 1.47 min; tR (AA+) ) 1.55 min. H NMR (500 MHz, MeOH-
1
H-N-Me-D-Ala-Tle-P r o-(R)-tetr ah ydr on aph th -1-ylAm ide
(16). tR (TFA+) ) 1.45 min; tR (AA+) ) 1.56 min. 1H NMR
(300 MHz, D2O) δ 7.17-7.35 (m, 4H), 4.99 (m, 1H), 4.61 (s,
1H), 4.39 (m, 1H), 3.70-4.09 (m, 3H), 2.80 (m, 2H), 2.67 (s,
3H), 2.30 (m, 1H), 1.72-2.15 (m, 7H), 1.57 (d, 3H), 1.08 (s,
9H). MS m/z ) 443 (M+H)+. HRMS for [C25H39N4O3]+ 443.3022;
found 443.3040.
d4) δ 8.15 (m, 1H), 7.89 (m, 1H), 7.78 (m, 1H), 7.45-7.58 (m,
4H), 4.75 (m, 1H), 4.69 (s, 1H), 3.99 (m, 2H), 3.78 (m, 1H),
2.68 (s, 3H), 2.40 (m, 1H), 2.20 (m, 2H), 2.03 (m, 1H), 1.52 (d,
3H), 1.10 (s, 9H). MS m/z ) 439 (M+H)+.
H-N-Me-Ala -Tle-P r o-n a p h th -2-yl Am id e (7). tR (TFA+)
1
) 1.62 min; tR (AA+) ) 1.71 min. H NMR (500 MHz, MeOH-
d4) δ 8.20 (m, 1H), 7.77 (m, 3H), 7.54 (dd, 1H), 7.35-7.47 (m,
2H), 4.69 (s, 1H), 4.59 (m, 1H), 3.96 (m, 2H), 3.80 (m, 1H),
2.67 (s, 3H), 2.33 (m, 1H), 1.95-2.21 (m, 3H), 1.48 (d, 3H),
1.13 (s, 9H). MS m/z ) 439 (M+H)+.
Exp r ession a n d P u r ifica tion of BIR3 P r otein . The BIR3
protein was prepared as described.24 NMR samples contained
2
2
0.8 mM protein in either 90% H2O/10% H2O or 100% H2O,
10 mM phosphate (pH 7.2), 120 mM NaCl, and 2 mM
2H-dithiothreitol.
H-N-Me-Ala -Tle-P r o-flu or en -9-yl Am id e (8). tR (TFA+)
1
) 1.62 min; tR (AA+) ) 1.73 min. H NMR (500 MHz, MeOH-
d4) δ 7.05-7.60 (m, 8H), 5.90 (s, 1H), 4.53 (s, 1H), 4.27 (t, 1H),
3.80 (m, 2H), 3.65 (m, 1H), 2.58 (s, 3H), 1.83-2.30 (m, 4H),
1.42 (d, 3H), 1.12 (s, 9H). MS m/z ) 477 (M+H)+.
H-N-Me-Ala -Tle-P r o-p h en eth yl Am id e (9). tR (TFA+) )
1.42 min; tR (AA+) ) 0.97 min. 1H NMR (300 MHz, MeOH-d4)
δ 7.15-7.32 (m, 5H), 4.63 (m, 1H), 4.32 (m, 1H), 3.90 (m, 2H),
3.70 (m, 1H), 3.34-3.55 (m, 2H), 2.80 (m, 2H), 2.66 (s, 3H),
1.75-2.15 (m, 4H), 1.46 (d, 3H), 1.09 (s, 9H). MS m/z ) 417
(M+H)+.
H-N-Me-Ala -Tle-P r o-(R)-tetr a h yd r on a p h th -1-yl Am id e
(2) was resynthesized in 12 mmol scale utilizing the standard
procedures for peptide coupling and Boc-removal to provide
4.97 g of the final product as the HCl salt (87% yield; (R)-1-
amino-1,2,3,4-tetrahydronaphthalene, Boc-Tle-OH, Boc-N-Me-
Ala as the coupling components). Compounds 10-16 were
synthesized analogously to compound 2 using the building
blocks shown in the Table 6.
NMR Sp ectr oscop y. All NMR experiments were acquired
at 303 K on a Bruker DRX600 NMR spectrometer. Resonances
were assigned using both nuclear Overhauser enhancement
spectroscopy (NOESY) and total correlation spectroscopy
(TOCSY) experiments.25 The protein side chain 1H and 13C
NMR signals were assigned from both HCCH-TOCSY experi-
ments and 3D 13C- or 15N-edited NOESY experiments. Protein
resonances assignments were made with the aid of our earlier
assignments for the Bir3/Smac peptide complex.11 Stereospe-
cific assignments of the valine and leucine methyl groups were
derived from our earlier Bir3/Smac peptide complex.11 Ligand
assignments and constraints were obtained from a 2D 13C,15N-
filtered NOESY and 2D 13C,15N-filtered TOCSY experi-
ment.26,27 NOE distance restraints (protein-protein and pro-
tein-ligand) were obtained from three-dimensional 13C-
edited,15N-edited, and 13C,15N-filtered 13C-edited NOESY
experiments26,27 acquired with
a mixing time of 80 ms.

Antagonists of the Antiapoptotic Protein XIAP
J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18 4425
Ligand-ligand constraints were obtained from the 2D 13C,15N-
ssas, VA) and were grown to passage 3 prior to inoculation.
The cells were mixed with Matrigel (BD Biosciences, Bedford,
MA) at a 1:1 ratio and 5 × 106 cells were inoculated into the
right flank of female scid mice (C.B-17-Prkdcscid, Charles River
Laboratories, Wilmington, MA). Tumors were measured by
digital calipers and the formula V ) LW2/2 was used to
estimate tumor size. Thirteen days after inoculation, mice were
assigned to treatment groups (N ) 10 per group) with an
average volume of approximately 230 mm3 per group. Tumor
size was then monitored by twice weekly measurements. The
Wilcoxon Rank Sum test was used to compare tumor size
between groups. The BIR3 ligands were both formulated in
0.1 M PBS, pH 5-6.
filtered NOESY experiment with a mixing time of 100 ms.
Str u ctu r e Ca lcu la tion s. BIR3/ligand structures were
calculated using a docking protocol implemented with the
program Xplor-NIH.28,29 Ligands were docked into the previ-
ously determined BIR3/SMAC average structure using rigid
body docking of the ligand and the protein followed by energy
minimization of the protein and ligand structures to optimize
the conformation of the complex. In the final refinement stage,
the protein and ligand conformation were refined based on the
NMR derived constraints. A square-well potential (FNOE )
50 kcal mol-1) was employed to constrain NOE-derived dis-
tances. On the basis of the cross-peak intensities, NOE-derived
distance restraints for the protein were given upper bounds
of 3.0, 4.0, or 5.0 Å, while the intermolecular distance
restraints were given upper bounds of 3.5 or 5.0 Å. Eight/ten
intraligand NOES and 95/108 intermolecular NOEs were
obtained for ligand 2/10, respectively. Protein-protein NOE
restraints were reassigned and quantitated for protein resi-
dues in the vicinity of the ligand resulting in 39/41 constraints
for ligand 2/10, respectively. Restraints for the remainder of
the protein were derived from our earlier BIR3/SMAC peptide
structure. This resulted in an additional 477 unambiguous
protein-protein NOE restraints, 36 hydrogen-bond restraints,
and 86 φ and æ angular restraints derived from the Talos
program that were used in the docking calculations. Super-
position of the protein backbone for the 10 lowest energy
structures resulted in an rmsd of 0.10, 0.08 Å for the heavy
atoms of ligands 2 and 10, respectively.
Ack n ow led gm en t. The authors would like to thank
J ennifer Bouska, Paul Richardson, and Xenia Searle for
technical assistance. We also thank the Abbott High-
Throughput Purification group and the Structural Chem-
istry group.
Su ppor tin g In for m ation Available: Stereoviews of NMR-
derived structures of compounds 2, 10, and AVPI bound to the
BIR3 domain of XIAP; BIR3 binding data for compounds from
the first library; toxicity of BIR3 ligands against 59 human
cancer cell lines. This material is available free of charge via
the Internet at http://pubs.acs.org.
Refer en ces
Liga n d Bin d in g. A fluorescence polarization anisotropy
(FPA) competition assay was used to determine the affinities
of compounds to BIR3 protein.11 Fluorescence polarization
measurements were carried out on an Analyst 96-well plate
reader (LJ L, Molecular Dynamic, Sunnyvale, CA). 6-Carboxy-
fluorescein (FAM) labeled peptides with the sequences AVPI-
AQK(FAM)-NH2 and AVPFAAK(FAM)-NH2 were used as
probes. To measure the potent compounds more accurately, a
tight-binding probe was designed and synthesized by substi-
tuting the position 2 residue Tle of compound 10 with a Lys
residue that bears a FAM label at the ꢀ-amino group. This
probe binds to the BIR3 protein with Kd of 7 nM. Dissociation
constants were determined from titration curves with in-house
written software using the analytical expressions of Wang.30
(1) Deveraux, Q. L.; Reed, J . C. IAP family proteins-suppressors of
apoptosis. Genes Dev. 1999, 13, 239-252.
(2) Tamm, I.; Kornblau, S. M.; Segall, H.; Krajewski, S.; Welsh, K.;
Kitada, S.; Scudiero, D. A.; Tudor, G.; Qui, Y. H.; Monks, A.;
Andreeff, M.; Reed, J . C. Expression and prognostic significance
of IAP-family genes in human cancers and myeloid leukemias.
Clin. Cancer Res. 2000, 6, 1796-1803.
(3) Krajewska, M.; Krajewski, S.; Banares, S.; Huang, X.; Turner,
B.; Bubendorf, L.; Kallioniemi, O. P.; Shabaik, A.; Vitiello, A.;
Peehl, D.; Gao, G. J .; Reed, J . C. Elevated expression of inhibitor
of apoptosis proteins in prostate cancer. Clin. Cancer Res. 2003,
9, 4914-4925.
(4) Srinivasula, S. M.; Hegde, R.; Saleh, A.; Datta, P.; Shiozaki, E.;
Chai, J .; Lee, R. A.; Robbins, P. D.; Fernandes-Alnemri, T.; Shi,
Y.; Alnemri, E. S.
A conserved XIAP-interaction motif in
caspase-9 and Smac/DIABLO regulates caspase activity and
apoptosis. Nature 2001, 410, 112-116.
Recon stitu ted Ca sp a se Activa tion Assa ys. Apaf-1 (4.0
nM), Csp-9 (p35/p12) or procaspase-9 (4.0 nM), 60 ng of procsp-
3, 200 µM dATP and 600 nM cytochrome C were preincubated
with 100 nM BIR3 or 5 nM XIAP in a final volume of 20 µL at
30 °C for 30 min. Increasing amounts of compound or SMAC
were then added to a final volume of 40 µL. Following
incubation for 30 min at 30 °C, DEVDamc cleavage activity
was assessed. Activity is expressed as an EC50, the concentra-
tion at which half-maximum recovery was achieved.
(5) Shiozaki, E. N.; Chai, J .; Rigotti, D. J .; Riedl, S. J .; Li, P.;
Srinivasula, S. M.; Alnemri, E. S.; Fairman, R.; Shi, Y. Mech-
anism of XIAP-mediated inhibition of caspase-9. Mol. Cell 2003,
11, 519-527.
(6) Chai, J .; Shiozaki, E.; Srinivasula, S. M.; Wu, Q.; Datta, P.;
Alnemri, E. S.; Shi, Y.; Dataa, P. Structural basis of caspase-7
inhibition by XIAP. Cell 2001, 104, 769-780.
(7) Huang, Y.; Park, Y.; Rich, R. L.; Segal, D.; Myszka, D. G.; Wu,
H. Structural basis of caspase inhibition by XIAP: differential
roles of the linker versus the BIR domain. Cell 2001, 104, 781-
790.
Cell Via bility. Human cancer cell lines were cultured in
suitable culture media containing 10% FBS. Adherent cell lines
were plated in 96-well plates and allowed to adhere for 24 to
48 h prior to compound exposure. Suspension cell lines were
used without preincubation. Cells were exposed to various
concentrations of compound for 48 or 72 h, and viability was
measured by MTS reduction with a commercial kit (Promega,
Madison, WI, CellTiter 96 Aqueous Assay). To assess protec-
tion from BIR3 ligand toxicity by the pan-caspase inhibitor
IDN-5370, cells were exposed for 48 h to 10 µM BIR3 ligand
and various concentrations of IDN5370, followed by measuring
viability.
Ca sp a se-3 Activity. Cells were lysed on ice in 0.1% Triton
X-100, 10 mM Hepes, 42 mM KCl, 5 mM MgCl2, pH 7.4, 1
mM DTT, and protease inhibitors (BD Biosciences, Franklin
Lakes, NJ , Cat #554779) after compound exposure and
caspase-3 activity was measured in 20 mM Hepes, 1 mM
EDTA, 0.1% CHAPS, 10% sucrose, 10 mM DTT, pH 7.5 with
20 µM Ac-DEVD-amc as substrate by following the release of
aminomethylcoumarin.
(8) Riedl, S. J .; Renatus, M.; Schwarzenbacher, R.; Zhou, Q.; Sun,
C.; Fesik, S. W.; Liddington, R. C.; Salvesen, G. S. Structural
basis for the inhibition of caspase-3 by XIAP. Cell 2001, 104,
791-800.
(9) Du, C.; Fang, M.; Li, Y.; Li, L.; Wang, X. Smac, a mitochondrial
protein that promotes cytochrome c-dependent caspase activa-
tion by eliminating IAP inhibition. Cell 2000 102, 33-42.
(10) Verhagen, A. M.; Ekert, P. G.; Pakusch, M.; Silke, J .; Connolly,
L. M.; Reid, G. E.; Moritz, R. L.; Simpson, R. J .; Vaux, D. L.
Identification of DIABLO, a mammalian protein that promotes
apoptosis by binding to and antagonizing IAP proteins. Cell
2000, 102, 43-53.
(11) Liu, Z.; Sun, C.; Olejniczak, E. T.; Meadows, R. P.; Betz, S. F.;
Oost, T.; Herrmann, J .; Wu, J . C.; Fesik, S. W. Structural basis
for binding of Smac/DIABLO to the XIAP BIR3 domain. Nature
2000, 408, 1004-1008.
(12) Wu, G.; Chai, J .; Suber, T. L.; Wu, J . W.; Du, C.; Wang, X.; Shi
Y. Structural basis of IAP recognition by Smac/DIABLO. Nature
2000, 408, 1008-1012.
(13) Chai, J .; Du, C.; Wu, J .-W.; Kyin, S.; Wang, X.; Shi, Y. Structural
and biochemical basis of apoptotic activation by Smac/DIABLO.
Nature 2000, 406, 855-862.
(14) Arnt, C. R.; Chiorean, M. V.; Heldebrant M. P.; Gores G. J .;
Kaufmann, S. H. Synthetic Smac/DIABLO peptides enhance the
effects of chemotherapeutic agents by binding XIAP and cIAP1
in situ. J . Biol. Chem. 2002, 277, 44236-44243.
Xen ogr a ft Effica cy Stu d ies. MDA-MB-231 cells were
obtained from the American Type Culture Collection (Mena-

4426 J ournal of Medicinal Chemistry, 2004, Vol. 47, No. 18
Oost et al.
(15) Guo, F.; Nimmanapalli, R.; Paranawithana, S.; Wittman, S.;
Griffin, D.; Bali, P.; O’Bryan, E.; Fumero, C.; Wang, H. G.;
Bhalla, K. Ectopic overexpression of second mitochondria-derived
activator of caspases (Smac/DIABLO) or cotreatment with
N-terminus of Smac/DIABLO peptide potentiates epothilone B
derivative-(BMS 247550) and Apo-2L/TRAIL-induced apoptosis.
Blood 2002, 99, 3419-3426.
(16) Yang, L.; Mashima, T.; Sato, S.; Mochizuki, M.; Sakamoto, H.;
Yamori, T.; Oh-Hara, T.; Tsuruo, T. Predominant suppression
of apoptosome by inhibitor of apoptosis protein in nonsmall cell
lung cancer H460 cells: therapeutic effect of a novel polyargi-
nine-conjugated Smac peptide. Cancer Res. 2003, 15, 831-837.
(17) Fulda, S.; Wick, W.; Weller, M.; Debatin, K.-M. Smac agonists
sensitize for Apo2L/TRAIL- or anticancer drug-induced apoptosis
and induce regression of malignant glioma in vivo. Nat. Med.
2002, 8, 808-815.
(22) Huang, Y.; Rich R. L.; Myszka, D. G.; Wu, H. Requirement of
both the second and third BIR domains for the relief of X-linked
inhibitor of apoptosis protein (XIAP)-mediated caspase inhibition
by Smac. J . Biol. Chem. 2003, 278, 49517-49522.
(23) Deckwerth, T. L. et al. Long-term protection of brain tissue from
cerebral ischemia by peripherally administered peptidomimetic
caspase Inhibitors. Drug Dev. Res. 2001, 52, 579-586.
(24) Sun, C.; Cai, M.; Meadows, R. P.; Xu, N.; Gunasekera, A. H.;
Herrmann, J .; Wu, J . C.; Fesik, S. W. NMR structure and
mutagenesis of the third Bir domain of the inhibitor of apoptosis
protein XIAP. J . Biol. Chem. 2000, 275, 33777-33781.
(25) Clore, G. M.; Gronenborn, A. M. Multidimensional heteronuclear
nuclear magnetic resonance of proteins. Methods Enzymol. 1994,
239, 349-363.
(26) Fesik, S. W.; Zuiderweg, E. R. P. J . Magn. Reson. 1988, 78, 588-
593.
(18) Vucic, D.; Kaiser, W. J .; Miller, L. K. Inhibitor of apoptosis
proteins physically interact with and block apoptosis induced
by Drosophila proteins Hid and Grim. Mol. Cell Biol. 1998, 18,
3300-3309.
(27) Marion, D.; Driscoll, P. C.; Kay, L. E.; Wingfield, P. T.; Bax, A.;
Gronenborn, A. M.; Clore, G. M. Overcoming the overlap problem
in the assignment of 1H NMR spectra of larger proteins by use
of three-dimensional heteronuclear 1H-15N Hartmann-Hahn-
multiple quantum coherence and nuclear Overhauser-multiple
quantum coherence spectroscopy: application to interleukin 1
beta. Biochemistry 1989, 29, 6150-6156.
(28) Brunger, A. T. X-PLOR Version 3.1, Yale University Press, New
Haven and London, 1992.
(29) Schwieters, C. D.; Kuszewski, J . J .; Tjandra, N.; Clore, G. M.
The Xplor-NIH NMR molecular structure determination pack-
age. J . Magn. Reson. 2003, 160, 65-73.
(19) Kipp, R. A.; Case, M. A.; Wist, A. D.; Cresson, C. M.; Carrell,
M.; Griner, E.; Wiita, A.; Albiniak, P. A.; Chai, J .; Shi, Y.;
Semmelhack, M. F.; McLendon, G. L. Molecular targeting of
inhibitor of apoptosis proteins based on small molecule mimics
of natural binding partners. Biochemistry 2002, 41, 7344-7349.
(20) Bellier, B.; McCort-Tranchepain, I.; Ducos, B.; Danascimento,
S.; Meudal, H.; Noble, F.; Garbay, C.; Roques, B. P. Synthesis
and Biological Properties of New Constrained CCK-B Antago-
nists: Discrimination of Two Affinity States of the CCK-B
Receptor on Transfected CHO Cells. J . Med. Chem. 1997, 40,
3947-3956.
(30) Wang, Z.-X. An exact mathematical expression for describing
competitive binding of two different ligands to a protein mol-
ecule. FEBS Lett. 1995, 360, 111-114.
(21) Zou, H.; Yang, R.; Hao, J .; Wang, J .; Sun, C.; Fesik, S. W.; Wu,
J . C.; Tomaselli, K. J .; Armstrong, R. C. Regulation of the Apaf-
1/caspase-9 apoptosome by caspase-3 and XIAP. J . Biol. Chem.
2003, 278, 8091-8098.
J M040037K