Strawberry Perl does not include Apache, but otherwise is wonderful. You can install Apache afterwards to run CGI like you want through a webserver on your local computer. But you have to set them up and make them work together.
It sounds like you already have some version of Apache running but are having trouble in the process of setting them up and getting perl and the server to work together.
If you are wanting begin learning Perl and CGI on a Windows platform, then a perfect way to do so is with IndigoPerl. It includes a great pre-configured (rather out of date) but ready to run Apache webserver for Windows.
You simply install it, and the Apache server is up and running on your localhost including perl programs.