Greetings everyone!

In today’s post I’m going to show you how you can get 99% modeling quality when backtesting using MT4’s “Strategy Tester.” If you’re a MT4 backtesting veteran you no doubt know that using regular M1 historic data the highest modeling quality you can get is 90%, but what you may or may not know is that the accuracy of your backtests can be pushed even higher, BUT you need raw tick data.

How does one obtain such tick data? Well, I asked myself that very same question, and I’ve looked everywhere for a free source of historical forex tick data and after extensive searching I found it. The only problem is that the tick data is not readily usable by MT4 and as such requires a rather elaborate process of conversion and also a few other tricks to force MT4 to use the converted tick data. That’s where this tutorial or “how-to” comes in. So pay close attention as you MUST follow each step precisely.

You may be asking yourself why would one want to get 99% modeling quality. The answer has to do with scalping. When backtesting an expert advisor that does not attempt to scalp the market the default MT4 history data provides an acceptably accurate level of simulation. However when you’re trying to backtest a scalping EA even the slightest variation in the price feed can strongly affect performance.

Before I move on I’d like to mention that a large portion of this tutorial was taken from Birt’s EA Review website, so please thank him by dropping by this website, subscribe to his RSS feed, etc. The conversion scripts are his too! Massive kudos to Birt for writing them.

Ok, now let’s move on. First off we need the forex tick data, and so far the best free source I’ve found is provided by the Swiss forex broker know as Dukascopy. This broker has a pretty good reputation from what I read on various internet forums and review sites, but for the moment our only main interest is getting our hands on their high quality tick data. Unlike many other brokers or data feed providers, Dukascopy offers FREE tick data for most most currency pairs spanning from April 2007 to now. Even though the tick data is being offered at no cost, it’s commercial-quality data and it approaches the ideal of 99% modeling quality. I’ve been able to get 99% modeling quality in my backtesting on a very consistent basis. The tick data is updated hourly directly from the Dukascopy data feed server. The bottom line is that this kicks ass! Now let’s hurry up and take advantage of this wonderful gift!

Necessary Tools:

Since the data is downloaded in files that span only 1 hour, some tools are necessary to download and parse it. These tools are a basically a collection of PHP scripts. Feel free to do whatever you want with the code, but don’t forget to give credit where it’s due – meaning to Birt.

You can download the script archive here.

You will find 3 scripts inside:

• A script for downloading the Dukascopy data, suggestively named “download_dukascopy_data.php”. As a courtesy for Dukascopy who is graciously providing free data, even though the script doesn’t download existing data, it still requests missing files, so, to avoid stressing their server, please set the dates in the $currencies array at the beginning of the script to the date of the last download; they’re using the standard linux seconds-since-1970 format, look up mktime to figure it out.
• A script for processing the downloaded data, which assumes that it’s in the same directory with the previous script and that the data was downloaded there (process_dukascopy_data.php); this one needs some parameters, run it without any for a description or check out the next script
• A small linux shell script that will process all the downloaded data

A minimum requirement to run these scripts is a command line PHP with the CURL extension. If you’re using a windows box you’re going to need to download the windows version of PHP. Don’t worry though as a download link will be provided.

Windows download & convert to CSV how-to

Start by going to the windows PHP download section and fetching the latest binary version as a zip file.
Once you’ve done that, unpack it to c:\php\ and also unpack the scripts from dukascopy_php_scripts.zip in the same directory.
Rename c:\php\php.ini-development to c:\php\php.ini.
Edit c:\php\php.ini, search for
;extension=php_curl.dll
and remove the semicolon in front of the line and add an “ext/” in front of “php_curl.dll” so that it looks like this:
extension=ext/php_curl.dll
Save the file and exit.
Edit 21.04.2010: if you run into a zip error and your PHP installation has an ext/php_zip.dll, apply the above for extension=ext/php_zip.dll.
Click start->run and type
cmd
then click ok (or alternatively type
cmd
then hit enter in the windows 7/vista “search programs and files” box in the start menu).
Type
cd \php
in the command window.
Type
php download_dukascopy_data.php
Have a coffee. Have another coffee. Go sleep. Go to work. Go to the gym. Go to a club. Wait some more. I’m not kidding, it takes ages. You can check the progress by watching the currency pair directories get filled. You can change the array at the beginning of the file to switch the order or to remove some pairs you don’t want. The number next to each pair is the timestamp at which to start downloading. If you get any strange errors, run the process again when it’s finished – it will only download any files that were missed in the first step due to network errors.
Edit 23.11.2009: if you want to easily convert from a regular date to a unix timestamp, you can use Epoch Converter, a very easy to use online tool.
When the download is finished, assuming you wanted to get the EURUSD data up to 01.11.2009, you’d type
php process_dukascopy_data.php EURUSD 200702 200911 EURUSD.csv
and the output will be placed in EURUSD.csv.
Alternatively, you can type
process.bat
which will batch process all the currency data. It’s mostly safe to ignore the error spam at this step.
This should be it, if everything went ok you should have your CSV files in the same c:\php folder.
Warning: make sure you have enough space on your harddisk. The downloaded tick files have over 7.5 GB while the CSV files have close to 60 GB at the time of this writing.

Using the tick data

Now, in order to use the CSV files, you need to perform some steps

1. Convert the data to FXT & HST
2. Copy the FXT & HST to the correct folders
3. Run MT4 with a loader to allow loading your custom FXT file, then start backtesting

I’ll go through each step below:

Converting the data to an MT4 format

To convert the data to an FXT file (and the accompanying HST files) usable in Metatrader 4, you need a new script that you can download here.

This script is based on GainData2fxt.mq4, an older script by an anonymous author and on FXTHeader.mqh, written by stringo@codebase.mql4.com.

This script has been updated to allow switching the GMT offset of the resulting FXT & HST files and to allow exporting the real spread in the file.

You have to perform the following steps in order to convert the CSV data:

• Copy the files from the dukascopy_mql_scripts_22.04.2010.zip archive to your MT4 installation. There’s a directory structure inside, make sure they land in the proper places
• Move the tick data file (the CSV file) to experts\files
• Open a chart for the pair that you have data for (if you have EURUSD.csv you MUST open an EURUSD chart)
• Select the timeframe that you wish to generate the FXT for (if you want to backtest on H1, then select H1 as the chart timeframe)
• Double click the Dukascopy2FXT script in the navigation panel (it’s in the scripts section) and configure the parameters in the window that pops up.
• Configure the parameters.
  For CsvFile you don’t need to write anything if the file is named just like the symbol and has a CSV extension; otherwise, just type the file name there.
  Select “true” for CreateHst only if you want the script to create HST files. You do not need to perform this step more than once (leave it on “false” if you’re regenerating data for a different timeframe), because if this parameter is set to true, history will be generated for the whole span of the CSV file and for all timeframes, regardless of the start/end date you configure (note that the FXT file is still limited by those).
  By using the Spread parameter, you can configure the desired spread of the FXT file. If left at 0, it will use the current spread that your broker has. A special note here would be that many brokers are widening their spreads during the weekends, so pay close attention. If set to anything other than zero, it will use the number as the spread; if your broker is a 5 digit broker, you have to enter 20 for a spread of 2 pips, while if your broker is a 4 digit broker, you would just have to enter 2. Small trick to change the spread without regenerating an FXT: open the existing FXT file with a hex editor; the spread is at offset 0xFC.
  The StartDate and EndDate fields control the span of the FXT file. There is a programming design bug in MT4, where the client is unable to read FXT files larger than 2GB and seeing that the tick data is rather huge, it’s very likely that your resulting file might be larger than that if you don’t pay attention. I added a “block” so the creation of the FXT file stops at 2GB, but keep in mind that if you don’t use set limits, you will probably not get the resulting data range that you desire. A good rule of the thumb is to use ~12 month periods for EURUSD and 16 month periods for any other currency. If the currency CSV file is below 2GB, most likely your resulting FXT file will be below 2GB as well and you’re probably safe to generate it for the whole range available in the CSV. The format of the StartDate and EndDate fields is YYYY.MM.DD and if left blank the script will just use the earliest, respectively the latest date available in the CSV file.
• Click “Ok”. Once you do that, the data generation process will start and it will typically take 5 to 30 minutes, depending on the data range and volume, possibly even more if you’re using a slow machine. When the processing is finished, you will get an alert and some data will be printed in the Experts log.
• Some people experienced miscellaneous issues. If your particular case is not listed here, feel free to drop a comment, I might be able to help.
  Among the common problems that the people encounter it’s worth mentioning the Windows 7 / Windows Vista users that run MT4 from their Program Files folder and that are unable to find the resulting files. The issue is caused by the fact that they have UAC enabled and these operating systems use folder virtualization. For Windows 7, the resulting files can be found in c:\ProgramData\ typically, while for Windows Vista they can be found in c:\Users\username\AppData\Local\VirtualStore\Program Files\. If they’re not there, just search for *.FXT, they must be somewhere. To get around this problem, either copy your MT4 folder in a location that’s not protected by UAC or simply disable UAC (type UAC in the start->search box thing and follow the on-screen process).
  Another issue experienced by some users is the lack of a MIN_LOT and LOT_STEP for the symbol in the resulted FXT file. This is caused by generating the FXT file with a MT4 client that has never been connected to a broker. When generating a file, do that on an MT4 client that has been connected to your broker at least once. Aka don’t use a “virgin” installation that has never seen a demo account.

Copying the data to the correct location

After the script above has finished running, you will end up with a lot of files in your “experts\files” folder. At this point, exit the MT4 terminal then proceed to moving any .HST files from “experts\files” to “history\your_server_name”. Pay very close attention if you have multiple server directories in your “history” folder – you will have to move them into the one that’s correct for active account!
Note: unless you modify the CSV or get a new one, the HST files only need to be generated and moved once.
After moving the HST files, move the generated FXT file from “experts\files” to “tester\history”.

Using the data in a backtest

Since the recent MT4 clients no longer offer the ability to use custom FXT files, a special loader is needed. You can download it here, it works only with the MT4 version commonly known as 225 (version 4.00 build 225).
To use the files you created in a backtest, you will have to copy the exe inside fxt_loader.zip into the folder of your MT4 client and instead of running terminal.exe, run “fxt loader.exe”. If you’re worried about running executable files and you can do a manual patch, here are the offsets and data:

hex offset -> old byte -> new byte
1402FC -> EE -> 00
1402FD -> 02 -> 00
14041B -> 9D -> 00
14041C -> 01 -> 00
140521 -> 97 -> 00

What the loader does is disabling the MT4 default behavior to regenerate the backtest files every time you start a new test – so, if a file is already there, a new one will not be generated. By placing the FXT file you generated with the script in the tester\history folder, you ensure that it’s the file it is going to use for your backtest; of course, the selected backtest currency pair and timeframe must coincide with what you generated the FXT with.

As an alternative to the loader above, you can use this loader which, in addition to what the other one does, will also let you run multiple instances of the terminal from the same installation folder. Read the comments below for more info and the patched offsets.

Also there is a new build 226 loader. It also includes an MQL script for the people who don’t want to run EXE files or who have particularly dumb antivirus packages; if you want to use the script version, start MT4 then put the script on any chart to enable the FXT functionality. However, keep in mind that while you can start more than an instance of the same MT4 client with the loader, you can’t do so with the script. If you want the patch offsets, they can be found in the script. Note that running the script on a build other than 226 (225 for example) will surely make it crash, so double check what build you’re running before you use it. On a separate note, you can make a similar script for build 225 by using the offsets posted above.

How to verify it worked?

Easy! If your test is running and the file in tester\history was not recreated, it means everything is OK and you’re in business!

GMT offset

This is a sensitive issue for many EAs. The Dukascopy data seems to be GMT-based all around the year, so the GMT offset that you need to use is 0. As of 21.04.2010, the Dukascopy and Oanda MQL scripts have an external parameter that lets you specify the GMT offset you would like for the resulting data, both HST and FXT.

Oanda tick data

If you’re one of the lucky people who can get the Oanda tick data (it’s only available to people who have an account there funded with over 1k USD), get it in CSV format and use the Oanda2FXT MQL script. It works just like the Dukascopy2FXT script described above.

Variable spread backtesting

If you want your backtest to be even more accurate, as of 21.04.2010 you also have the option to use the real spread from the tick data. All you have to do is:

1. Make sure you’re using the latest Dukascopy MQL script or the latest Oanda MQL script.

2. When running the script, set UseRealSpread to true.

3. When loading the terminal, use the variable spread loader (which is different from the other loaders).

Note that as of now, this only works on Metatrader 4 build 225. I don’t think I’ll be making one for 226 anytime soon, so if there’s anyone who fancies asm and thinks they can handle porting it to 226, here’s what I changed:

00401323 DA0D 36134000 FIMUL DWORD PTR DS:[401336]
00401329 DB17 FIST DWORD PTR DS:[EDI]
0040132B E9 26020000 JMP terminal.00401556

00401336 A0860100 //100000
0040133A ACC52737 //0.00001

00401556 DA27 FISUB DWORD PTR DS:[EDI]
00401558 D815 3A134000 FCOM DWORD PTR DS:[40133A]
0040155E EB 12 JMP SHORT terminal.00401572

00401572 DFE0 FSTSW AX
00401574 F6C4 01 TEST AH,1
00401577 E9 E5020000 JMP terminal.00401861

00401861 DC42 1C FADD QWORD PTR DS:[EDX+1C]
00401864 74 06 JE SHORT terminal.0040186C
00401866 DC83 20030000 FADD QWORD PTR DS:[EBX+320]
0040186C C3 RETN

00541209 E8 1501ECFF CALL terminal.00401323
0054120E 90 NOP

Downloads:

For easy reference here are all the files referred to in this post.

Dukascopy data download / parse scripts
Dukascopy MQL process scripts
MT4 build 225 FXT loader
MT4 build 225 FXT multiloader
MT4 build 225 FXT multiloader with variable spread
MT4 build 226 FXT multiloader and script
Oanda MQL Process script

I hope you found this tutorial useful! Thanks goes out once again to Birt for writing the PHP scripts and the how-to guide. If you wish to chat with other forex traders about MT4 backtesting in general or specifically about this method of achieving 99% modeling quality I strongly recommend you join the Forex Nirvana forum. The specific forum section dedicated MT4 backtesting is located here:

http://www.forexnirvana.com/f41/

Happy backtesting everyone!

Cheers,

Alan

Okay, so you’ve set up your history data and completed your backtest using Metatrader Strategy Tester. Now let’s look at the results.

There are four tabs to look through: Report, Graph, Results, and Journal

Report:

This is the meat of the backtest. All the statistics of the backtest are displayed here.

• Bars In Test – Displays the depth of the history on which the modeling was based.

• Ticks Modelled – Displays the size of the modeled sequence. Every record of the sequence represents the bar state (OHLCV) at one or another moment. Different bar states can be modeled depending on the timeframe, modeling method, and presence of history data from smaller timeframes within a bar.

• Modelling Quality – Calculated according to the following formula. This should be 90%. If not, read the article on downloading and setting up history data.

ModellingQuality = ((0.25*(StartGen-StartBar) +
0.5*(StartGenM1-StartGen) +
0.9*(HistoryTotal-StartGenM1)) / (HistoryTotal-StartBar))*100%;

• Gross Profit – The summed up profit for all profitable transactions;

• Gross Loss – The summed up loss for all unprofitable transactions;

• Total Net Profit – The Gross profit minus the Gross loss.

TotalNetProfit = GrossProfit – GrossLoss

• Profit Factor – Profit factor, shows the ratio between gross profit and gross loss. The larger this number is, the better.

        ProfitFactor = GrossProfit / GrossLoss

• Expected Payoff – Calculated as follows:

        Expected Payoff = (ProfitTrades / TotalTrades) * (GrossProfit / ProfitTrades) –
(LossTrades / TotalTrades) * (GrossLoss / LossTrades)

• Absolute Drawdown – Absolute drawdown is the difference between the initial deposit and the smallest value of balance within testing:

        AbsoluteDrawDown = InitialDeposit – MinimalBalance

• Maximal Drawdown – The highest difference between one of local upper extremums of the balance graph and the following lower extremums:

        MaximalDrawDown = Max of (Maximal Peak – next Minimal Peak)

The rest of the statistics are pretty self explanatory, so we won’t waste time defining those. For more information on this, visit Metatrader’s website on backtesting

Graph:

This is pretty obvious, it is a graph of the Expert advisor’s trade results. The green lines at the bottom mark the lot sizes.

Results:

This tab displays every order placed with the running total of your account balance.

Journal:

This displays any errors and status messages.

Original article source: The Trade Machine (c)

http://thetrademachine.com/blog/2009/10/02/analyzing-backtest-results-in-metatrader-strategy-tester/

Metatrader’s Strategy Tester allows you to optimize your expert advisor to find the most profitable settings for the multitude of possible input variable combinations. You can optimize an expert advisor on a specific chart for a specific time period to find out which input values produced the highest returns and lowest drawdown.

However, it is important to remember that you should not over-optimize. You may get amazing results with a specific input combination, but because the values are so constrained, you probably won’t see results like that in the future. When optimizing an expert advisor, remember to use your head and think about why you are getting the results you are. Was it from one huge winner that you are unlikely to see in the future? Or did it win many times, making it a plausible trade setup for the future.

Don’t forget, you need to have high quality minute data for optimization to be worthwhile. See the article on setting up your metatrader history data.

How To Optimize – Step By Step

To get started optimizing your expert advisor, open Strategy Tester. If you are new to backtesting, read the article: How to Backtest in Metatrader

Select the Expert Advisor from the pull down menu. Choose a currency Symbol and Period. Select Every Tick for Model. Check Use Date and select a time frame that you have high quality minute history data for. Finally check the box next to Optimization.

Next, click on the Expert Properties button on the right. Select the INPUTS tab to enter the range of values you will optimize for. The Start value is the lowest value that you start from. Stop is the highest. And Step is the increment that you step through the range.

For the TakeProfit value, you can see that we start at 50, step through at 25, and stop at 300. This will test every value inbetween 50 and 300 in a 25 increment. So, 50, 75, 100, 125, all the way up to 300. The same idea applies to the other variables.

The checkbox on the left must be checked to optimize for that variable. Otherwise, it will use the regular Value.

When you’re finished entering your input values, click OK. Now we’re ready to start the optimization. Click the Start button on the bottom right of the Strategy Tester window. The backtest may take minutes or hours depending on how many values you are optimizing for. You may want to choose a larger increment Step value or a smaller range if it is taking too long.

After the optimization has completed, select the Optimization Results tab to look through the results. You can sort by the values at the top. Select Profit to view in order of highest returns, or Drawdown to see which values had the least risk.

Original article source: The Trade Machine (c)

http://thetrademachine.com/blog/2009/10/08/metatrader-optimization/

So you’ve just gotten your hands on a hot new expert advisor and you need to know if this thing is really going to work. It’s time to get your hands dirty and start testing. There are two types of testing you should do: forward testing and backtesting.

But first, make sure you have high quality minute data for your backtest. Read this article on Downloading and Setting up History Data to Get 90% Backtesting Quality.

Forward Testing:
Forward testing simply means to open a demo account and run the expert advisor in real time to see how it works and performs. Forward testing is essential, however it takes a lot of time before you get a good idea about how the expert advisor will perform. Instead of relying solely on forward testing, we can also perform backtests.

Backtesting:
Backtesting lets you test years of data in just minutes. So, before we decide to spend a lot of time forward testing, we can use Metatrader’s Strategy Tester to see how it will perform long term.

Strategy Tester also allows you to optimize your expert advisor. See the optimization article on this.

History Data

Before you start backtesting you will need to make sure you have complete and accurate history data set up. If your modeling quality is less than 90% or you see mismatched chart errors, then your data is insufficient.

To download history data and learn how to set it up, read this article: How To Set Up Metatrader History Data And Get 90% Backtesting Quality

Strategy Tester Backtest Settings

Now that you have your history data set up, let’s start backtesing!

To open the Metatrader backtesting window, select from the top menu: VIEW > STRATEGY TESTER (or Ctrl + R for the shortcut). The window should open in the bottom of the terminal and look something like this:

Most of the Strategy Tester input values are pretty self explanatory. First, select your Expert Advisor and the Symbol of the currency pair you want to test.

For Model enter Every Tick (the most precise method based on all available least timeframes to generate each tick). This mode allows to model price movement within a bar the most precisely.

For Period enter the time frame you want.

Leave Optimization unchecked for the moment.

Check Use Date and select the start time and end time that you want to test. Make sure you have quality history data for this time period.

Select Visual Mode only if you want a visually step through the backtesting. Go ahead and try this to see how it works. It is a good tool for checking the logic of your expert advisor and to spot any errors. However, it does take a little more time to perform, so you might not want to always use this.

Next, click on the Expert properties button on the right.

Select the TESTING tab. Enter an Initial Deposit value and leave everything else the same.

Select the INPUTS tab. This is where you enter any input values for your expert advisor. These will vary depending on your own EA. Change what you want, or leave the default settings.

Don’t worry about the OPTIMIZATION tab for now. Click OK to close the box.

Now you are ready to start your back test. Hit the START button and let’s see what results we get. Depending on your EA and computer memory, it may take several minutes to hours to complete.

Once your backtest has completed, it is time to Analyzing the backtest results

Original article source: The Trade Machine (c)

http://thetrademachine.com/blog/2009/10/01/backtesting-with-metatraders-strategy-tester/

Setting up Metatrader 4 for backtesting can be a tricky, time consuming, and a little frustrating if you aren’t entirely sure you have it all working properly. This article will guide you through the entire process from obtaining high quality minute history data to installing a separate backtesting account to importing and converting the tick data to different time frames. In no time you will be up and backtesting years of history data with 90% modeling quality!

Step 1: Download high quality minute (M1) Alpari history data for Metatrader. This link takes you to Megaupload where the file is hosted. Save and extract the data into a folder for future use.

Step 2: Download a new installation of Metatrader from Metaquotes.net. This will be your testing version of Metatrader. Install it in a new folder, making sure not to copy over any existing installations of Metatrader. I’ve named my new version MetaTrader 4 Tester 2.

Step 3: When Metatrader installer asks you if you want to launch Metatrader upon completing the installation, choose to NOT launch Metatrader by UNCLICKING the check box. Then select finish the installation.

Step 4: After you have finished installing a new version of Metatrader 4, delete all the folders within your Metatrader history folder. To find this folder browse the Metatrader program file where it was installed. This is typically in the directory: Program Files > Metatrader 4 (or whatever you chose to name this) > history.

Step 5: Now that you have deleted all of the history files, launch Metatrader 4 Terminal and create a demo account with whatever broker they offer you. You should be prompted to create this demo account when you first open Metatrader. If you are not, don’t worry. Just right click ACCOUNTS on the left and choose to OPEN AN ACCOUNT.

Step 6: Once your new demo account has been created, you will need to DELETE it. Yes, you read that correctly. We need to delete the demo account we just opened. We do this to prevent the demo account from overwriting the history data that we will be importing in upcoming steps. You are probably asking: Then why did we open this demo account to begin with? The reason is to create a folder structure for all the currency pairs, which we will be importing data into very soon.

Step 7: After you delete your new account, close Metatrader. Now we are going to take the Alpari history data we downloaded and extracted earlier and move it into our Metatrader Demo account folder. Browse to the history folder within your Metatrader folder as we did earlier. You should see a folder of the demo account we created and then deleted earlier (Note: if you can’t see this folder and you are running Windows Vista, you may need to change some settings in Vista before you can see this folder. You can also skip this step and just import the history files from wherever they currently are in the upcoming steps.)

Before we copy over the M1 history data, make sure there aren’t any left over .hst files from your demo account. If there are delete those.

Now copy over all of the history data files to this demo folder. Copy over just the files and not the folder structure so that you end up with all the data files together in the demo folder like this:

Step 8: Next, re-launch the Metatrader terminal. From now on, every time you launch Metatrader, you will be prompted to “Open An Account”. Choose CANCEL each time you are asked this in the future.

You will also be asked to login to your account. Click CANCEL for this one each time too.

Step 9: Now we need to change the chart settings to their maximum values so that they can hold all the tick data we will be importing. With Metatrader open, select TOOLS > OPTIONS.

Choose the CHARTS tab and edit the “Max Bars In History” and “Max Bars In Chart” to their maximum values by entering as many ‘9’s as it will let you. Click OK.

The next time you open this options menu these max values will be readjusted to the highest number of bars allowed:

Step 10: Next, go to TOOLS > HISTORY CENTER (or press F2 for the shortcut).

On the left hand side find the currency pair that you want to import data for. We have chosen AUDUSD. Expand the menu so that you can see the different time frames. They should be grayed out at the moment. Double click on the 1 Minute timeframe of your pair to select it. It will stay grayed out, but you need to double click it so that we can import the minute data.

Step 11: Click IMPORT. Then click BROWSE and go to the Metatrader > history > demo account directory where we placed the minute data earlier. Select the minute data and open it. You may need to change File Type from ASCII Text to All Files so that you can see the data to open.

You should see the data populate the import box. It may take a few moments to load. If it still doesn’t load, you may need to enter a ‘1’ for Skip Columns. Or you if the data is not separated by commas, change it to whatever format it is in. For ours, the only thing we changed was Skip 1 Column. Once the data is loaded, hit OK.

Because the data is separated into 2 files of different time periods for each pair, you will need to import each file like we did above. So, after we import the data for AUDUSD 2001-2006, we will import again for the other AUDUSD file “M1_AUDUSD”. Once we complete this, we will have a complete time period from 2001 to 2009.

Repeat these steps of importing data for each pair. Once you have finished, hit OK to exit the History Center.

Step 12: Open a chart for a pair that you imported data into. Select the Minute timeframe (M1) at the top. You should see a chart of the minute data for the time period that you imported. All other time frames should still say “Waiting for update” because we still have to convert the minute data into all those time frames.

Step 13: Converting The Data – From the Navigator window on the left, expand SCRIPTS and open period_converter. (TIP: Make sure you have the EXPERTS tab selected at the bottom of the terminal so you can view the status of this script.) You will need to convert the minute data to each time frame by changing the ExtPeriodMultiplier value and running the script. M1 is already done, so let’s start with the 5 minute. Enter 5 for M5 and click OK.

You will know it has finished converting when the EXPERTS tab displays a message that it has written the records.

Open period_converter again. It will ask you: “Do you really want to stop period_converter”. Choose Yes and repeat for the following time frames: Enter 15 for M15, 30 for M30, 60 for H1, 240 for H4, 1440 for D1, and 10080 for W1.

Step 14: Repeat the above steps for each currency pair.

Step 15: That’s it. Now that you have your history data all set up, you are ready to Start Backtesting With Metatrader’s Strategy Tester

Original article source:  The Trade Machine (c)

http://thetrademachine.com/blog/2009/09/29/set-up-metatrader-history-data-get-90-backtesting-quality/

MetaTrader4 User Guide:

This is a LONG post so be prepared to do a LOT of scrolling (hope you have a wheel mouse.) If you do not feel like reading this whole blog post through and would instead prefer an offline version of this guide please see this link.

Here is a picture of what the MT4 main screen looks like:

• Main menu (access to the program menu and settings);
• Toolbars (quick access to the program features and settings);
• “Market Watch” window (real-time quotes for the specified instruments);
• “Navigator” window (quick access to all your accounts, indicators, expert advisors);
• “Chart” window;
• “Terminal” window (quick access to the trading and account history, list of open positions and placed orders, client terminal log-file and internal mailbox).

Download any demo MetaTrader4 installation program from the broker of your choice. This guide uses FXDD’s MT4 demo platform as a visual guide but this does not mean this guide and/or website recommends or is affiliated with this broker.

1. Launch the Metatrader exe file the same way you launch any Windows program. It’s best if you select to save to disk.

2. Scroll down the list to select the setup language. Then press the Next button:

3. Click the Next button after reading the notice:

4. In the next window check the “Yes, …” box if you agree with the terms of the license agreement. Then press the Next button:

5. Select data path and press the Next button:

6. Then use the default program group for FXDD and press the Next button:

7. Press the Next button to install MetaTrader 4 on your computer:

8. Installation is in process:

9. Once FXDD has been installed click the “Launch FXDD – MetaTrader 4 box.
Then press the Finish button:

10. First-time FXDD users will be prompted to fill in the application form to open a demo account: You must click the “I agree to subscribe into your news letters” box in order to launch the program. Then click the Next button.

11. Use the default server and click Next.

12. You will see your login and password to trade on the demo account.

Click Finish to gain access to your demo account.

Note, that if your demo account is inactive for 90 days it is automatically
deleted by the server.

 13. Your demo account platform will start up and display the following window. Click Start to update your platform.

Then click Update and Restart to restart your platform.

The Market Watch Window:

 The “Market Watch” window can be activated by pressing the Ctrl+M key combination, or by using the View->Market Watch menu sequence or by clicking the button from the “Standard” toolbar:

Right click the mouse button on the “Market Watch” window to produce the context menu:

In MetaTrader 4, you can see as many real time charts as you wish, all on one screen, for any currency pair and for any time period. The following types of charts are available in MetaTrader 4:

• Line Chart. Chart x-axis is for the different time periods (minutes, hours, days, months etc.), and y-axis is for the price. The point on the chart shows the close price for the period, i.e. the bid of the last quote for this period. All the sequential points are joined by lines:

It is recommended that this chart type is used only for the short time periods (up to 5 minutes). In order to change the chart to a line chart, press the button on the «Charts» toolbar:

or press the Alt+3 key combination or use the Charts -> Line Chart menu sequence.

• Bar Chart. The chart is created with the use of bars. Each bar has a high (top), a low (bottom), an opening price (left horizontal little line) and a closing price (right horizontal little line) for the specified period of time (for example, an hour):

It is recommended that you use this chart type for periods of 5 minutes or more. In order to change the chart to a bar chart press the button on the “Charts” toolbar

or press the Alt+1 key combination, or use the Charts -> Bar Chart menu sequence.

• Candlesticks. These charts are created in the same way as bar charts:

Candlesticks charts were very popular in the 80-90s because of their clear graphics. This method of technical analysis is based on the correlation between both closing and opening prices within the same trading day, and the closing price of the previous session and the opening price of the next session.

Candlesticks have different configurations compared to bar charts. The rectangular part of the candlestick line is called the real body. The thin vertical lines above and below the body are called shadows. They show lows and highs within a specified period of time. Depending on the closing and opening prices the real body can be white (empty) or black. When the body is white, the closing price is higher than the opening level. When the body is black, the closing price is lower than the opening level.

In order to create a candle chart press the button on the “Charts” toolbar:

or press the Alt+2 keys simultaneously, or use the Charts -> Candlesticks menu sequence.

In MetaTrader 4 both the bar and candle high is the highest Bid price for the defined period, the low is the lowest Bid price, the closing price is the last quoted Bid, the opening price is the first quoted Bid.

In order to create a new chart for any instrument right click on the “Market Watch” window, then select the Chart window item from the context menu:

By left clicking the mouse you can drag and drop any selected currency pair into any open chart window.

Chart period – the period displayed in one bar or candle. For example, if you create a 15 minute bar chart, then each bar will have the price data for the relevant 15 minutes. In MetaTrader 4 the following time periods can be used: 1, 5, 15, 30 minutes (M1, M5, M15, M30 respectively), 1 hour (H1), 4 hours (H4), 1 day (D1), 1 week (W1) and 1 month (MN). In order to change a time period, use the “Periodicity” toolbar:

Right click anywhere on the chart to produce the context menu to get access to the chart settings:

• Choose Periodicity and select a time period from the drop down menu or press the button on the “Charts” toolbar;

• Select Refresh item if an error occurs or if there are so called “gaps” on the chart. All missing data will be uploaded from the server and will appear on the chart;

• Select Grid if you wish to hide/unhide gridlines on the chart;

• Select Volumes in order to show/hide tick volume (the number of quotes of any period time);

• Zoom in / Zoom out will help you to modify chart measures. You can also use the / buttons on the “Charts” toolbar or press the “+” and “-” buttons;

• Save as picture… will save the chart as the *.gif, or *.bmp. file;

• Properties… (or the F8 button) enables the window where you can change chart settings, by selecting the “Color” tab you can change the color of any element on the chart:

Other settings can be changed in the Common tab:

Please note the following options:

• Offline chart – the chart is not being updated by new quotes. Once you open the chart offline (File -> Open offline) this option becomes automatic;

• Chart shift – allows you to shift the chart from the right edge of the window, to see the most recent candle forming. You can also press the button on the “Charts” toolbar or use the Charts -> Chart shift menu sequence;

• Chart autoscroll – autoscrolls to the left once a new quote is received. You can also press the button on the “Charts” toolbar or use the Charts -> Autoscroll menu sequence. If you are viewing the chart back a few days, and the autoscroll button is depressed, it will automatically take you to the most recent candle forming on the next tic of the price.

Open a position:

In order to enable the “Order” window:

• right click on the “Market Watch” window or on the “Trade” tab of the “Terminal” window and select New order from the context menu; or

• press the F9 button; or

• use the Tools -> New order menu sequence; or

• right click on the chart window and select Trading -> New Order from the context menu; or

• double click on the currency pair in the “Market Watch” window.
  

The “Order” window:

In MetaTrader 4 there are two ways of making deals: “Instant Execution” and “Pending Order”.

Instant Execution means you do not have to request quotes, and you will enter the market immediately. Once you have decided to buy or sell at the current price press the Buy or Sell buttons to open a long or short position.

Before you enter the market you can change the details in the “Order” window:

• Symbol – a currency pair

• Volume – number of lots;

• Stop Loss – the level of Stop Loss order;

• Take Profit – the level of Take Profit order;

• Comment – any notes (that will show up in your report)

The new position will now be listed in the “Trade” tab of the “Terminal” window.

Close a position:

In order to close a position, highlight the trade in the “Trade” tab of the “Terminal” window and right click to obtain the context menu:

then select Close order. Double clicking on the open position in the “Trade” tab of the “Terminal” window will have the same result. The “Order #” window appears and you click the Yellow Close bar.

Place/Modify/delete Stop Loss & Take Profit Orders

Stop Loss – an order to close a previously opened position at a price less profitable for the Customer than the price at the time of placing the order

Take Profit – an order to close a previously opened position at a price more profitable for the Customer than the price at the time of placing the order.

If you place these types of orders you do not have to sit in front of your computer to wait for the outcome. Take Profit and Stop Loss orders will be executed automatically as soon as the price reaches the order level (see below). You can place Stop Loss and Take Profit at the time of making a deal filling in the “Stop Loss” and “Take Profit” fields in the “Order” window:

Afterwards in order to modify Stop Loss and Take Profit levels use the context menu:

Right click on the open position or on the pending order in the “Trade” tab of the “Terminal” window then select Modify or delete order. The “Modify order” window enables:

• Fill in the “Stop Loss” and “Take profit” fields. By pressing the red button (in our example “1.2121″) you can copy this level into the “Stop Loss” field. By pressing the blue button (in our example “1.2096″) you can copy this level in the “Take profit” field. The value in the “Level” field defines the difference between the order level and the current price. Once you fill in the “Stop Loss” or/and “Take profit” fields press the red “Modify #position number …” button.

  In order to delete Stop Loss or Take profit orders specify its level as equal to zero.

Trailing Stop: is an algorithm to manage Stop Loss orders. Once a trailing stop has been placed (for example, trailing stop’s value is 15 pips) the following occurs:

• No Stop Loss order will be placed until the open position is in a profit of at least 15 pips. MetaTrader 4 then sends an instruction to place the Stop Loss order 15 pips higher or lower than the current price.

• Subsequently, each time the difference between the current price and the order level exceeds 15 pips, MetaTrader 4 sends the instruction to modify the Stop Loss order to a level 15 pips higher or lower than the current price. As a result, the Stop Loss order becomes closer to the current price.

Important:
It is very important to note that no Stop Loss order will be placed until the market moves in your favor as mentioned above. Therefore until this has happened, to limit losses, a separate Stop Loss would need to be placed manually.

In order to place a trailing stop, highlight the open position in the “Trade” tab of the “Terminal” window and right click to open the context menu.

Pending Order: is an instruction to open a position when the current price reaches the order level.
There are four types of pending orders:

• Buy Stop – an order to open a Buy position at a price higher than the price at the moment of placing the order;

• Sell Stop – an order to open a Sell position at a price lower than the price at the moment of placing the order;

• Buy Limit – an order to open a Buy position at a lower price than the price at the moment of placing the order;

• Sell Limit – an order to open a Sell position at a price higher than the price at the moment of placing the order.

In order to place a pending order, open the “Order” window by any way described below:

• right click on the “Market Watch” window or on the “Trade” tab of the “Terminal” window and select New order from the context menu; or

• press the F9 button; or

• use the Tools -> New order menu sequence; or

• right click on the chart window and select Trading -> New Order from the context menu; or

• double click on the currency pair in the “Market Watch” window.

Then choose the Pending order item from the drop down

The following options may be changed in this window as well:

• Symbol – currency pair;

• Volume – transaction size in lots;

• Stop Loss – Stop Loss order level which will be activated once the pending order has been executed;

• Take Profit – Take Profit order level which will be activated once the pending order has been executed;

• Comment – your comments for the pending order;

• Type – the type of pending order: Buy Stop, Buy Limit, Sell Stop, Sell Limit;

• At price – pending order level;

• Expiry – if the box has not been checked the order is valid up to the time when you cancel it or when it is executed. If the box is checked the order is valid up to the moment specified in the box.

Once the Place order button is pressed it appears in the “Trade” tab of the “Terminal” window.

In order to modify or delete a pending order highlight it in the “Terminal” window, then right click to
produce the context menu:

Select the Modify or Delete Order menu item. Press the Modify button or the Delete button depending on your choice in the dialog window:

That does it for this guide. I hope it has been of use to you. Thanks for reading my blog.

Cheers,
Alan
http://alansforexblog.com

Hot keys (accelerating keys) are keys and their combinations that allow you to execute various commands fast and without the use of menus or toolbars. Here is a lit of some very useful hot keys for the MT4 platform:

• “” — chart scrolling to the left;

• “” — chart scrolling to the right;

• “” — fast chart scrolling to the left or, if the scale is defined, chart scrolling up;

• “” fast chart scrolling to the right or, if the scale is defined, chart scrolling down;

• Page Up — fast chart scrolling to the left;

• Page Down — fast chart scrolling to the right;

• Home — move the chart to the start point;

• End — move the chart to the end point;

• “-” — chart zoom out;

• “+” — chart zoom in;

• Delete — delete all selected graphical objects;

• Backspace — delete the latest objects imposed into the chart window;

• Esc — close the dialog window;

• F8 — call the chart Properties window;

• F9 — call the “New Order” window;

• F11 — enlarge and minimize the chart window;

• Ctrl+B — call the “Objects List” window;

• Ctrl+F — enable “Crosshair”;

• Ctrl+G — show/hide grid;

• Ctrl+I — call the “Indicators List” window;

• Ctrl+P — print the chart;

• Ctrl+Y— show/hide period separators;

• Ctrl+Z — undo an object deletion;

Cheers,

Alan

http://alansforexblog.com

MT4 Backtesting & Optimization

Here is a visual tutorial (aka HOWTO) on how to run optimizations and backtests with MetaTrader 4 (MT4). This is but one method of backtesting! There is another method that you can use but which involved downloading backtesting data NOT from your broker but from MetaQuotes (the makers of MetaTrader 4 – MT4) directly. For a guide on how to use this other method please see this post of mine instead.

I hope you find the guide useful and if you do feel free to give me a shout out by leaving a comment. Here goes the guide:

*Due to the limited screen real estate my current blog template affords I had to reduce the size of the images a bit. So if you find that you cannot clearly read a particular item in this visual guide please click on any of the images for a full screen version*

By the way in case you are wondering the EA’s backtested in this guide come from Bogie Enterprises.

That about does it for this post.

Have fun backtesting! Hey don’t forget if you find that elusive holy grail EA, send me a copy!

Cheers,
Alan

http://alansforexblog.com

Hi everyone. Here is an interesting tutorial that I found on a forum (Google search is your friend.) I hope the admin does not mind me reproducing his post available @ http://forexbarometer.com. I hope you find it useful. Here comes the article:

Spread is one of the crucial parameters that might dramatically affect short-time trading systems performance. By such systems we assume TS that hunts for 8-20 pips profits and keeps opened positions no longer than for 1-2 hours.

The big MT4 platform inconvenience is dependence of the spread used in tester from current or last market conditions. For instance, in Friday market closed with 8 pips spread that is twice bigger than usual 4 pips. During weekend, every time we run tester on historical data, it will use the same last known 8 pips spread and we cannot optimize our TS for normal market conditions of 4-5 pips.

One well known solution is editing a spread value in *.FXT binary file used by tester but since MT4 build 210 tester rewrites this file before running a test/optimization process. So, to implement this *.FXT- tweaking technique we must use build 208 or 209. Those versions tester has a “Recalculate” checkbox that allows to prevent binary data file regeneration and keep our spread intact. The older builds might not work with your broker (Old version error in connection window).

Therefore, to begin testing our trading system with variable spread we need:

• Older TERMINAL.EXE (main MT4) file v.208-209. It’s better to have a complete installation package because older and newer components may be incompatible. E.g. EA compiled in 223 version will crash under 208.
• Tweaking utility. Generally, it may be any hex-editor like WinHex or special utlility like FXB Tester History Editor (FXB THE). The latter is free for all FXB customers and will be uploaded to common customers’ area next week.
• TS *EX4 file(s) compiled for used older version (208 or 209). This is a very important issue because any TS will crash on incompatible software unless you have a source code for that and are able to recompile it.

These are test results for Leader family TS performed on IBFX mini-account for period of April 1-10, 2009. Tests made in Saturday, April 11, when market is closed and MT4 keeps last low liquidity market spread of eight pips, that is definitely unacceptable value for both testing/optimizing and trading.

*Note: In Firefox you can right click and select “View Image” to view the full image*

–end of article–

Thanks for your attention. I wish you all profitable trading!

Alan
http://alansforexblog.com

Greetings dear fellow forex traders. Today I’m going to briefly show how you can get your MetaTrader account history performance statement to show the profit or loss in PIPs. I’m pretty sure there must be a few of you out there who’ve always wondered how to get MT4 to do this. Unfortunately the bad news is that MT4 by default cannot do this. What is required is a little bit of work.

What you gotta do is open the statement file with MS Excel (technically it should work with other spreadsheet programs as well) and use the following function:

=IF(AND(F6<20;C6=”buy”);(J6-F6)*10000;IF(AND(F6<20;C6=”sell”);( F6-J6)*10000;IF(AND(F6>20;C6=”buy”);(J 6-F6)*100;(F6-J6)*100)))

or

=IF(AND(F6<20,C6=”buy”),(J6-F6)*10000,IF(AND(F6<20,C6=”sell”),( F6-J6)*10000,IF(AND(F6>20,C6=”buy”),(J 6-F6)*100,(F6-J6)*100)))

Here it is in action:

*If you’re curious to know, that’s a sample image I found on the net, it’s not my screenshot – I’m not Greek*

That’s it! Hopefully the next version of MetaTrader – aka MetaTrader 5 (MT5) – supports this ability by default (MetaQuotes are you listening?) because in my opinion it is a very useful feature for those of us who think in pips instead of dollars.

Ok, that about does it for this tutorial. I hope you found it useful. If so give me a shout (leave a comment) letting me know

Cheers,
Alan
http://alansforexblog.com