[WiX-users] Any documentation on WiX/Windows Installer limits?

Discussion:

Alex Lian

2007-01-29 03:29:56 UTC

I've been googling for the spec somewhere (as well as about the tools)

What's the file size limitations of wix? I seem to be able to create the
.msm with 4 files
totalling 3.4GB though it seems unnaturally compressed down to 805MB.

However, in trying to create a final .msi using this .msm, yields an
E_ABORT error trying
to access the msm file. Orca can seem to open it fine, but I'm having
trouble trying to
re-extract the files at the moment.

So...

any pointers to docs/info on limits?

I'm sure I'm still doing something amiss, and I'll keep struggling on
it.

-Alex

Rob Mensching

2007-01-29 04:07:45 UTC

Permalink

Cabinet files (the compression technology used by the Windows Installer) can currently only handle up to 2GB. Unfortunately, with Merge Modules you only get one cabinet. However, for MSI files you can split your product into multiple cabinets (separate Media elements) to get around the problem.

-----Original Message-----
From: wix-users-***@lists.sourceforge.net [mailto:wix-users-***@lists.sourceforge.net] On Behalf Of Alex Lian
Sent: Sunday, January 28, 2007 7:30 PM
To: wix-***@lists.sourceforge.net
Subject: [WiX-users] Any documentation on WiX/Windows Installer limits?

I've been googling for the spec somewhere (as well as about the tools)

What's the file size limitations of wix? I seem to be able to create the
.msm with 4 files
totalling 3.4GB though it seems unnaturally compressed down to 805MB.

However, in trying to create a final .msi using this .msm, yields an
E_ABORT error trying
to access the msm file. Orca can seem to open it fine, but I'm having
trouble trying to
re-extract the files at the moment.

So...

any pointers to docs/info on limits?

I'm sure I'm still doing something amiss, and I'll keep struggling on
it.

-Alex

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Alex Lian

2007-01-30 17:03:12 UTC

Permalink

Ahh...that makes sense (with also additionally experimenting by me
confirming some of that).
So, I'm still experimenting - splitting up the files across cabs/media -
but light.exe
seems to choke on the large MSI it outputs in the temp directory.

I see this error:
light.exe : error LGHT0001 : Failed to open database: C:\<path
removed>\Temp\jjua6pkf\<filename removed>.msi

Exception Type: System.Runtime.InteropServices.ExternalException

Stack Trace:
at Microsoft.Tools.WindowsInstallerXml.Msi.Database.Open(String path,
OpenDat
abase type)
at Microsoft.Tools.WindowsInstallerXml.Binder.GenerateDatabase(Output
output)

at Microsoft.Tools.WindowsInstallerXml.Binder.Bind(Output output)
at Microsoft.Tools.WindowsInstallerXml.Tools.Light.Run(String[] args)

If I pare off some of the larger files (> 1GB each) it seems to be able
to create a ~2GB installer,
but it chokes when I add the next file. Any thoughts before I start code
diving in light.exe?
Is light.exe hindered at some file size?

-Alex

On Sun, 28 Jan 2007 20:07:45 -0800, "Rob Mensching"

Post by Rob Mensching
Cabinet files (the compression technology used by the Windows Installer)
can currently only handle up to 2GB. Unfortunately, with Merge Modules
you only get one cabinet. However, for MSI files you can split your
product into multiple cabinets (separate Media elements) to get around
the problem.
-----Original Message-----
Sent: Sunday, January 28, 2007 7:30 PM
Subject: [WiX-users] Any documentation on WiX/Windows Installer limits?
I've been googling for the spec somewhere (as well as about the tools)
What's the file size limitations of wix? I seem to be able to create the
.msm with 4 files
totalling 3.4GB though it seems unnaturally compressed down to 805MB.
However, in trying to create a final .msi using this .msm, yields an
E_ABORT error trying
to access the msm file. Orca can seem to open it fine, but I'm having
trouble trying to
re-extract the files at the moment.
So...
any pointers to docs/info on limits?
I'm sure I'm still doing something amiss, and I'll keep struggling on
it.
-Alex
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share
your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
WiX-users mailing list
https://lists.sourceforge.net/lists/listinfo/wix-users

Rob Hamflett

2007-01-30 17:11:50 UTC

Permalink

The programmer in me can't help but wonder if Light is using an int for the file size, instead of an
unsigned int. The former's top end is just over 2GB, and would explain what you're seeing. Whether
or not that's the problem, I don't know. It's just what immediately jumped out at me.

Rob

Post by Alex Lian
Ahh...that makes sense (with also additionally experimenting by me
confirming some of that).
So, I'm still experimenting - splitting up the files across cabs/media -
but light.exe
seems to choke on the large MSI it outputs in the temp directory.
light.exe : error LGHT0001 : Failed to open database: C:\<path
removed>\Temp\jjua6pkf\<filename removed>.msi
Exception Type: System.Runtime.InteropServices.ExternalException
at Microsoft.Tools.WindowsInstallerXml.Msi.Database.Open(String path,
OpenDat
abase type)
at Microsoft.Tools.WindowsInstallerXml.Binder.GenerateDatabase(Output
output)
at Microsoft.Tools.WindowsInstallerXml.Binder.Bind(Output output)
at Microsoft.Tools.WindowsInstallerXml.Tools.Light.Run(String[] args)
If I pare off some of the larger files (> 1GB each) it seems to be able
to create a ~2GB installer,
but it chokes when I add the next file. Any thoughts before I start code
diving in light.exe?
Is light.exe hindered at some file size?
-Alex
On Sun, 28 Jan 2007 20:07:45 -0800, "Rob Mensching"

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Rob Mensching

2007-01-30 17:39:32 UTC

Permalink

The FileSize column in the File table has a datatype of "DoubleInteger". From the documentation:

"The DoubleInteger data type is a four-byte integer value. Unless otherwise restricted, the range of legal values is from -2,147,483,647 to +2,147,483,647."

Also, it is a known issue that cabinet files cannot handle more than ~2 GB of data in a single cabinet.

-----Original Message-----
From: wix-users-***@lists.sourceforge.net [mailto:wix-users-***@lists.sourceforge.net] On Behalf Of Alex Lian
Sent: Tuesday, January 30, 2007 9:03 AM
To: wix-***@lists.sourceforge.net
Subject: Re: [WiX-users] Any documentation on WiX/Windows Installer limits?

Ahh...that makes sense (with also additionally experimenting by me
confirming some of that).
So, I'm still experimenting - splitting up the files across cabs/media -
but light.exe
seems to choke on the large MSI it outputs in the temp directory.

I see this error:
light.exe : error LGHT0001 : Failed to open database: C:\<path
removed>\Temp\jjua6pkf\<filename removed>.msi

Exception Type: System.Runtime.InteropServices.ExternalException

Stack Trace:
at Microsoft.Tools.WindowsInstallerXml.Msi.Database.Open(String path,
OpenDat
abase type)
at Microsoft.Tools.WindowsInstallerXml.Binder.GenerateDatabase(Output
output)

at Microsoft.Tools.WindowsInstallerXml.Binder.Bind(Output output)
at Microsoft.Tools.WindowsInstallerXml.Tools.Light.Run(String[] args)

If I pare off some of the larger files (> 1GB each) it seems to be able
to create a ~2GB installer,
but it chokes when I add the next file. Any thoughts before I start code
diving in light.exe?
Is light.exe hindered at some file size?

-Alex

On Sun, 28 Jan 2007 20:07:45 -0800, "Rob Mensching"

Alex Lian

2007-01-30 20:02:20 UTC

Permalink

I'm not exceeding any of those statements.
All files are *individually* < 2GB, and I'm setting up multiple cabs
such that none exceed 2GB. I'll double check my wxs file, but I'm pretty
sure I tried to verify that.
I got this error when I did exceed that:

light.exe : fatal error LGHT0017 : Failed to create cab '<path
removed>\Temp\kq9kbz9g\client_base.cab' while cabbing file '<filename
removed>' with error 0x80070020.

So, it seems more like I have problems when light tries to re-open the
msi file after adding the cabs in...

-Alex

On Tue, 30 Jan 2007 09:39:32 -0800, "Rob Mensching"

Post by Rob Mensching
The FileSize column in the File table has a datatype of "DoubleInteger".

Rob Mensching

2007-01-30 21:13:04 UTC

Permalink

0x80070020 is SHARING_VIOLATION. Do you have a virus scanner watching the built files? That's the way this happens most often. Virus scanners are notorious for locking up compressed files. I always exempt the build drives from our virus scanners.

-----Original Message-----
From: wix-users-***@lists.sourceforge.net [mailto:wix-users-***@lists.sourceforge.net] On Behalf Of Alex Lian
Sent: Tuesday, January 30, 2007 12:02 PM
To: wix-***@lists.sourceforge.net
Subject: Re: [WiX-users] Any documentation on WiX/Windows Installer limits?

I'm not exceeding any of those statements.
All files are *individually* < 2GB, and I'm setting up multiple cabs
such that none exceed 2GB. I'll double check my wxs file, but I'm pretty
sure I tried to verify that.
I got this error when I did exceed that:

light.exe : fatal error LGHT0017 : Failed to create cab '<path
removed>\Temp\kq9kbz9g\client_base.cab' while cabbing file '<filename
removed>' with error 0x80070020.

So, it seems more like I have problems when light tries to re-open the
msi file after adding the cabs in...

-Alex

On Tue, 30 Jan 2007 09:39:32 -0800, "Rob Mensching"

Post by Rob Mensching
The FileSize column in the File table has a datatype of "DoubleInteger".

Alex Lian

2007-01-31 01:13:38 UTC

Permalink

Ahh...whoops...I think I invoked a build at the same time as that
particular error.
But that still doesn't address the problem I consistently see when I
increase the overall MSI size (with multiple cabs of < 2GB size)
seems to be > 2GB...

filenames and paths removed to protect the innocent. =)

Microsoft (R) Windows Installer Xml Linker version 2.0.4820.0
Copyright (C) Microsoft Corporation 2003. All rights reserved.

Updating file information.
Generating database.
Merging modules.
Processing media information.
Cabbing file xxx from 'xxx'.
Cabbing file xxx from 'xxx'.
Cabbing file xxx from 'xxx'.
Cabbing file xxx from 'xxx'.
Cabbing file xxx from 'xxx'.
Cabbing file xxx from 'xxx'.
Cabbing file xxx from 'xxx'.
Cabbing file xxx from 'xxx'.
Importing streams.
Importing cabinet stream 'xxx.cab' from
'<pathremoved>\Temp\gblw976s\xxx.cab'.
Importing cabinet stream 'xxx.cab' from
'<pathremoved>\Temp\gblw976s\xxx.cab'.
Importing cabinet stream 'xxx.cab' from
'<pathremoved>\Temp\gblw976s\xxx.cab'.
light.exe : warning LGHT1011 : Access denied; cannot delete
'<pathremoved>\Temp\gblw976s'.
Warning, failed to delete temporary directory:
<pathremoved>\Temp\gblw976s
light.exe : error LGHT0001 : Failed to Commit database

Exception Type: System.Runtime.InteropServices.ExternalException

Stack Trace:
at Microsoft.Tools.WindowsInstallerXml.Binder.ImportStreams(String
databasePath, Output output)
at Microsoft.Tools.WindowsInstallerXml.Binder.Bind(Output output)
at Microsoft.Tools.WindowsInstallerXml.Tools.Light.Run(String[] args)

On Tue, 30 Jan 2007 13:13:04 -0800, "Rob Mensching"

Post by Rob Mensching
0x80070020 is SHARING_VIOLATION. Do you have a virus scanner watching
the built files? That's the way this happens most often. Virus scanners
are notorious for locking up compressed files. I always exempt the build
drives from our virus scanners.
-----Original Message-----
Sent: Tuesday, January 30, 2007 12:02 PM
Subject: Re: [WiX-users] Any documentation on WiX/Windows Installer limits?
I'm not exceeding any of those statements.
All files are *individually* < 2GB, and I'm setting up multiple cabs
such that none exceed 2GB. I'll double check my wxs file, but I'm pretty
sure I tried to verify that.
light.exe : fatal error LGHT0017 : Failed to create cab '<path
removed>\Temp\kq9kbz9g\client_base.cab' while cabbing file '<filename
removed>' with error 0x80070020.
So, it seems more like I have problems when light tries to re-open the
msi file after adding the cabs in...
-Alex
On Tue, 30 Jan 2007 09:39:32 -0800, "Rob Mensching"

Post by Rob Mensching
The FileSize column in the File table has a datatype of "DoubleInteger".

Bob Arnson

2007-01-31 04:48:58 UTC

Permalink

Post by Alex Lian
But that still doesn't address the problem I consistently see when I
increase the overall MSI size (with multiple cabs of < 2GB size)
seems to be > 2GB...

Have you tried using external cabs? My day job is working with a
multi-gigabyte package but the .msi itself is only ~70MB.

Post by Alex Lian
light.exe : warning LGHT1011 : Access denied; cannot delete
'<pathremoved>\Temp\gblw976s'.

That's a sign another process is using the directory.

--
sig://boB
http://bobs.org

Alex Lian

2007-01-31 13:41:28 UTC

Permalink

Post by Bob Arnson
Have you tried using external cabs? My day job is working with a
multi-gigabyte package but the .msi itself is only ~70MB.

Well, the hope was to get a single file install .msi, but I can see if
that makes a difference.

Post by Bob Arnson
That's a sign another process is using the directory.

Hmm...I thought I filemon'ed and checked who else tried to access the
directory/files.
I guess I'll try filemon'ing again.

Alex Lian

2007-01-31 17:37:39 UTC

Permalink

Well, I'm pretty sure now that light.exe has a 2GB limitation on
its own for the MSI file itself (regardless of using cabs < 2GB in
size).

I verified by a couple facts:

- Inspection of the temp folder showed multiple cabs, each < 2GB.
- After resulting error, .msi file is slightly under 0x80000000
(2147483648) in size.
- Filemon showed writing to the msi file up to slightly under the same
point at which
point it didn't even try to write to the msi, whereas it kept reading
cab files long after.
- The error

So, the long and short is that .msi files with the v2 wix tools seem
limited to 2GB.
Once I get a dev env setup at home, I'll start poking at it.

Bob Arnson

2007-02-01 01:27:37 UTC

Permalink

Post by Alex Lian
Well, I'm pretty sure now that light.exe has a 2GB limitation on
its own for the MSI file itself (regardless of using cabs < 2GB in
size).

Light passes most of the .msi construction on to MSI itself, so that's
likely where any limit lives. Structured storage files can theoretically
be larger than 2GB but there's no doc whether MSI itself supports it.
Such large packages probably aren't a common scenario.

--
sig://boB
http://bobs.org

Alex Lian

2007-02-01 21:13:37 UTC

Permalink

Post by Bob Arnson
Light passes most of the .msi construction on to MSI itself, so that's
likely where any limit lives. Structured storage files can theoretically
be larger than 2GB but there's no doc whether MSI itself supports it.
Such large packages probably aren't a common scenario.

Oh well, I guess that's the limitation then.
Once I get my dev box at home re-setup (yay re-install) I still feel
like taking
a gander.

In any case, I have to say it's an interesting limit they have, as it's
only
a matter of time that installers will creep pass this line regularly.
The installer
I was trying to experiment and replace is a 4GB install. Lots of data to
deploy, and
we were preferring the single file for download since it's then simpler
for the user.
If were only DVD installs, then breaking apart the cabs becomes an ok
option, but
with web downloads it looks less attractive.

I would point out that these big installers are not that uncommon a
scenario. I'm working
for a game developer currently, and in this industry *lots* of games are
regularly large installations.
Additionally, they're more regularly offering downloadable installs.
It's not uncommon to download >1.5GB demo games...so we're already close
to that line of 2GB.

Just thought I'd throw out that point, such that the Installer team can
keep that
in consideration. Especially, since MS seems to like games to offer good
installation
experiences, and with these size constraints it's forcing us to consider
other more
hacky options then.

-Alex

Bob Arnson

2007-02-06 16:15:28 UTC

Permalink

Post by Alex Lian
I would point out that these big installers are not that uncommon a
scenario. I'm working
for a game developer currently, and in this industry *lots* of games are
regularly large installations.
Additionally, they're more regularly offering downloadable installs.
It's not uncommon to download >1.5GB demo games...so we're already close
to that line of 2GB.

In my day job, I work with a ~9GB installer on two DVDs. We have an
800MB download demo but it's a self-extractor for download. Enough
products require a setup.exe chainer or other form of bootstrapper that
wrapping an .msi and .cabs together isn't a big problem.

Post by Alex Lian
Just thought I'd throw out that point, such that the Installer team can
keep that
in consideration.

I don't think anyone on the MSI team lurks on this list. But you can
send feature requests to ***@microsoft.com and MSI team members see
them.

--
sig://boB
http://bobs.org

Albert van Peppen

2007-01-31 11:54:54 UTC

Permalink

Hi,

I guess that will work.

I still believe the ~2GB limit is the problem here.

I cannot find the max filesize for System.IO.File but I recon its 64bit
(unsigned)? If it is 32bit (signed), then the ~2GB Limit is the
problem...

Usualy in VC++.NET 2005 you use Cfile, which is adapted to use ULONLONG
which is 64bit (unsigned).

But since I'm not a C# guru I place this question for another guru ;)

Another problem might be that you're temp files are on a FAT 32, or
worse a FAT 16, partition?

Regards,

Albert van Peppen

-----Oorspronkelijk bericht-----
Van: wix-users-***@lists.sourceforge.net
[mailto:wix-users-***@lists.sourceforge.net] Namens Bob Arnson
Verzonden: woensdag 31 januari 2007 5:49
Aan: Alex Lian
CC: wix-***@lists.sourceforge.net
Onderwerp: Re: [WiX-users] Any documentation on WiX/Windows Installer
limits?

Post by Alex Lian
But that still doesn't address the problem I consistently see when I
increase the overall MSI size (with multiple cabs of < 2GB size) seems
to be > 2GB...

Have you tried using external cabs? My day job is working with a
multi-gigabyte package but the .msi itself is only ~70MB.

Post by Alex Lian
light.exe : warning LGHT1011 : Access denied; cannot delete
'<pathremoved>\Temp\gblw976s'.

That's a sign another process is using the directory.

--
sig://boB
http://bobs.org

------------------------------------------------------------------------
-
Take Surveys. Earn Cash. Influence the Future of IT Join
SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE
V