[Ron_Jeremy]

Messing around in the registry, I noticed the option to "ClearPageFileAtShutdown". It is set by default to "0" (no). What is the purpose of enabling this feature? Are there any performance advantages/disadvantages in doing do?

[DosFreak]

Security.

The pagefile may contain the data that you were working on. Clearing it a shutdown makes it harder to find the data. I would not enable it. The pagefile will be remade on bootup re-fragmenting your files.

[jaywallen]

That registry entry is controlled by the Local Security Policy named "Shutdown: Clear virtual memory pagefile". It doesn't remove the pagefile, but merely wipes it. So the pagefile doesn't have to be re-created at boot if the option is turned on. It might be worthwhile option to enable if you keep encrypted data on the system and don't want anyone to be able to snoop the pagefile. If you don't use encrypted data, I'm not sure why anyone would bother to use it. If the pagefile is big, this option can make shutdown take quite a while.

Regards,
Jim

[jaywallen]

I'm not sure I see the point you're making. The pagefile is NOT deleted by this security setting. The contents are wiped. Fragmentation will not result from the use of this setting. That's all I was saying.

The pagefile and registry hives are defragged by the Sysinternals utility, Pagedefrag. However it doesn't touch the MFT or metadata. O&O makes a decent defragger that performs a defragging operation of all of this stuff at boot time, and in very little more time than it takes for Pagedefrag to run. However, the versions that do boot time defragging are not freeware.

Regards,
Jim

[jaywallen]

Quote:

Did I say it was deleted by clearing it above?

It can be deleted, by filesystem corruptions!

That wasn't the topic under discussion. I was merely trying to be certain that it was understood that the security setting being discussed would NOT delete the pagefile itself, and therefore would not result in a file system fragmentation issue, in and of itself.

Quote:

(I gain speed by housing the pagefile/swapfile onto another disk... on EIDE a second one on another EIDE I/O channel, & on ScSi on another drive device on the chain. So, when one drive is seeking/reading/writing for me? The swapfile & temp. operations take place on another.. simultaneously! Makes for good performance sense!)

* Understand now?

APK

P.S.=> You are bringing in the possibility of MFT$ defrags now? Diskeeper from Executive Software also does the same as well... not a freeware one, & not in their LITE versions either! I told folks abotu a FREEBIE they can use for PageFile & Reg file defrags above! apk

I pointed out the differences in cost in my own post. For the information of anyone who's interested in the differences, the Executive Software product has to be set each time to perform the boot time defrag, whereas the O&O product can be set to perform it automatically at each boot.

As for you, APK, you might want to have that ego checked. Your voluminous posts speak volumes about you but more, I think, about a presumptuous nature than about knowledge.

[jaywallen]

I don't really need confirmation of my opnions from others, but I've seen the comments about your "contributions", and they are far from unaniously slanted in your favor. Take a hint. You are presumptuous, and no one needs a doctorate in psychology to see that.

I guess you're at least relatively safe with your puffery online. Hard to get away with it in real life, isn't it?

[Ron_Jeremy]

O.K you two, let it go. Besides, you're both talking over my head now anyway. Thanks for initial replies - they're most appreciated. If you're gonna continue the mud slinging, I'm gonna delete this thread.

[jaywallen]

Ron_Jeremy,

Sorry for the unpleasantness.

Regards,
Jim

[ghayes]

"The pagefile and registry hives are defragged by the Sysinternals utility, Pagedefrag. However it doesn't touch the MFT or metadata. O&O makes a decent defragger that performs a defragging operation of all of this stuff at boot time, and in very little more time than it takes for Pagedefrag to run. However, the versions that do boot time defragging are not freeware."

Part of this statement is correct and part is incorrect. The correct part is that Sysinternals doesn't provide a mechanism to defragment the Master File Table ($MFT) or related metadata.

The incorrect part is that O&O's defragger will defragment the MFT and metadata. O&O defragments the $MFT only - it doesn't defragment the $Logfil, $Bitmap, $Upcase, etc... There is only 1 defragger available that will defragment these metadata files - PerfectDisk - it is also the only defragger that tells you how badly fragmented these metadata files are. Defraggers like O&O Defrag only tell you how badly fragmented the $MFT is.

- Greg/Raxco Software

Disclaimer: I work for Raxco Software, the maker of PerfectDisk - a competitor to O&O Defrag, as a systems engineer in the support department.

[clutch]

Well damn ghayes, took ya long enough to get here!

[jaywallen]

Quote:

The incorrect part is that O&O's defragger will defragment the MFT and metadata. O&O defragments the $MFT only - it doesn't defragment the $Logfil, $Bitmap, $Upcase, etc... There is only 1 defragger available that will defragment these metadata files - PerfectDisk - it is also the only defragger that tells you how badly fragmented these metadata files are. Defraggers like O&O Defrag only tell you how badly fragmented the $MFT is.

- Greg/Raxco Software

Disclaimer: I work for Raxco Software, the maker of PerfectDisk - a competitor to O&O Defrag, as a systems engineer in the support department.

Sorry, I should have been more careful / precise. Have you examined the "Select Additional Files" feature on the Boot Time Defragmentation dialog in O&O? Once you have performed one full defragmentation of a drive, you have the option to add the files that couldn't be defragged with the GUI online by using the Add Exclusive feature. I won't pretend to know whether or not that comprises all the metadata, but that is some or most of it, isn't it? I mentioned it because it's a feature that I've seen many users / evaluaters of O&O overlook. Anyway, once you add the exclusively locked files, they also get defragged at boot time.

In addition to the manual Action | Boot-Time Defragmentation settings, the Executive Software Product does have FragGuard which can be set to run when fragmentation exceeds certain levels on the MFT or registry hives (but without mention of any other items), but I didn't see evidence that it could defrag the "unmovable" files on an NTFS partition.

BTW, I tried out Perfect Disk about a year-and-a-half ago when I was evaluating defraggers for use with Win2K. (I've been using Windows only since a couple of months before the advent of Win2K.) I thought it was generally a good product, but I had some problems with the user interface on a notebook with an ATI graphics subsystem that I couldn't resolve with tech support and had to resort to O&O.

Regards,
Jim

Edit: I asked you if the "additional files" comprised any significant portion of the metadata but didn't tell you what they were. DOH! I'd be glad to PM or e-mail the list to you.

[ghayes]

Jim,

"Sorry, I should have been more careful / precise. Have you examined the "Select Additional Files" feature on the Boot Time Defragmentation dialog in O&O? Once you have performed one full defragmentation of a drive, you have the option to add the files that couldn't be defragged with the GUI online by using the Add Exclusive feature. I won't pretend to know whether or not that comprises all the metadata, but that is some or most of it, isn't it? I mentioned it because it's a feature that I've seen many users / evaluaters of O&O overlook. Anyway, once you add the exclusively locked files, they also get defragged at boot time."

I can state with utmost certainty that O&O Defrag does NOT do any of the metatdata besides the $MFT. Even if you go into the Boot Time defrag options and select Additional Files, you are not presented with a way to select any of these other metadata files from their interface (do you see a file called $MFTMir or $Logfile or $Upcase?).


AlecStaar:

Diskeeper also doesn't defragment these other metadata files. The interesting thing about Diskeeper is that even if the $MFT is actually in 1 piece, Diskeeper will always show it as being as in 2 pieces. Why? Because they count the $MFTMirr - one of the metadata files - as a fragment of the $MFT - even though it is a separate file.

This is easier to see on an NT4 NTFS partition.

- Go to a MSDOS prompt and go to the top level of a NTFS partition.
- Issue the following command:
Attrib $MFT
Attrib $MFTMirr
Attrib $Logfile

These are just 3 of the NTFS metadata files.

If you try to find out non-$MFT fragmentation information in any other defrag product, it can not be found.

The reason SpeedDisk can sometimes only get the $MFT down to 2 pieces is that SpeedDisk can't move the 1st records of the $MFT. This means that if the beginning of the $MFT is not at the top of the logical partition, then SpeedDisk has to leave it where it is - but may put the remainder of the $MFT at the top of the logical partition.

Even though I work for a competitor, I do know quite a lot about other defrag products and what they can and cannot do :-)

- Greg/Raxco Software

[ghayes]

A little bit of info about NTFS metadata...

NTFS is a self-describing file system. This means that all of the information needed to "describe" the file system is contained within the file system itself - in the form of metadata.

The $MFT is where all of the information about files are stored - in the form of file id's. A file ID is comprised of a 64bit number - of which 2/3 is the actual FileID and the remaining 1/3 is a sequence number. When files are deleted from an NTFS partition, the file id isn't immediately re-used. Only after hundreds of thousands of files are created is the sequence number incremented and the "empty" file id re-used. That is why the $MFT continues to grow and grow and grow. It is also why the $MFT Reserved Zone exists - to allow the $MFT to grow "into" it - hopefully in a contiguous fashion. Very small files can be stored "resident" in the $MFT. As much of the $MFT as can fit into memory is loaded when the partition is mounted.

The $MFTMirr is an exact copy of the 16 records of the $MFT. The first 16 records of the $MFT contain files 0 - 15. File 0 is the $MFT. File 1-15 are the remainder of the metadata (not all used btw...). The $MFTMirr is NTFS's "fallback" mechanism in case it can't read the 1st 16 records of the $MFT.

The $Bitmap is exactly that - a file containing a bit for each logical cluster on the partition - with the bit either being set or clear depending if that logical cluster is free or used.

The $Logfile is NTFS's transaction log - all updates to disk first go through the transaction log. This transaction log is what provides for NTFS's recovery (roll back/forward transactions) when the operating system is abnormally shutdown/crashes and provides for enhanced file system integrity.

$Upcase is used for Unicode information (foreign language support, etc...).

These are just a few of the NTFS metadata files and what they are used for. Windows 2000 introduced new metadata files (i.e. $Usnjnl and $Reparse).

Regarding SpeedDisk:

SpeedDisk is the only commercial defragger that does NOT use the defrag APIs provided by Microsoft as part of the NT/2000/XP operating system. These APIs are tightly integrated with the Windows Memory Manager, caching system and file system and take care of all of the low level I/O synchronization that has to occur to allow safe moving of files online - even if the files are in use by other users/processes. The APIs impose some restrictions, however. Pagefiles can't be defragmented online, (nor the hibernate file under Win2k), directories can't be defragmented online under NT4 (FAT and NTFS) and Win2k (FATx). The $MFT and related metadata can't be defragmented online as well. In order to get around these restrictions, SpeedDisk "wrote their own" stuff to move files - it has a filter driver that gets installed/run. This is why SpeedDisk can be service pack/hotfix dependent. Depending on the changes that MS makes to the Memory Manager and file system, SpeedDisk may have to be updated to safely run. That is why (for example), if you have Windows 2000/SP2 installed and run SpeedDisk, it displays a warning message about not being compatible with that version of the operating system and proceed at your own risk...

I know HOW SpeedDisk is doing what they are doing. However, knowing what can happen if they calculate things incorrectly, makes me a bit wary. However, SpeedDisk is alot better product - in terms of actually being able to normal data files - than some of the other defrag products out there.

- Greg/Raxco Software

[clutch]

Nice post Greg.

[EddiE314]

true, a nice read, but long as h.ell.

[DosFreak]

Diskeeper 7 can now defraf 4K+ clusters. (Took 'em long enough)

[DrPizza]

Quote:

Messing around in the registry, I noticed the option to "ClearPageFileAtShutdown". It is set by default to "0" (no). What is the purpose of enabling this feature? Are there any performance advantages/disadvantages in doing do?

The purpose is to permit NT to gain C2 security evaluation.

It has no notable impact, other than slowing shutdown times, and, of course clearing the pagefile.

It would be theoretically possible for sensitive information to be left in the pagefile, and hence recoverable. C2 has strict rules on the re-use of resources, and so to prevent this kind of behaviour it has this option available to you.

[DrPizza]

Quote:

Security.

The pagefile may contain the data that you were working on. Clearing it a shutdown makes it harder to find the data. I would not enable it. The pagefile will be remade on bootup re-fragmenting your files.

I don't believe this is true; it clears the pagefile, not deletes it (as I understand it; this was the NT 3.51 behaviour, and I don't see any reason for it to be different).

[DrPizza]

Quote:

Diskeeper 7 can now defraf 4K+ clusters. (Took 'em long enough)

It wasn't their fault.

The defrag FSCTLs provided by the NTFS driver didn't work for clusters greater than 4 kbytes.

[DrPizza]