How a query performs depends very much on the filtering conditions, such as you have in the WHERE clause here. It just may be that the first query is taking too long because the where clause is not selective enough. If you have access to te Oracle Server you can run the same query on it directly and see how many rows it returns and how long it takes to see if the problem is replicated there.
I finally decided to give up using a linked server to an Oracle DB since it takes too long to get a result. I believe it is not the appropriate thing to do when the table in Oracle DB is too huge. Using a ADO.NET connection type within a Visual Studio ASP.net project, works perfectly fine.