调用WebService下载大量文件遇到的问题及解决办法
【IT168 技术文档】
最近一个项目中,需要让每个镜像站点同步更新。我在服务器A生成文件,再调到BCDE四台服务器的WebService进行下载更新。
调动时把所以文件的地址作为一个字符串参数传递,最初的下载方法如下:
public static void DownloadFile(string uri, string filename) { WebClient wc = new WebClient(); wc.DownloadFile(uri, filename); }
下载少量文件没有出现问题,当文件数达到几万级时,就出现文件没有更新过去。跟踪错误显示WebRequest请求超时,这时想到设定
TimeOut的值来让它不超时,可是WebClient似乎没有这一属性,只得改用HttpWebRequest了,更改后如下:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(SiteDomain); request.Timeout = TIME_OUT; private bool DownloadFile(string uri, string FileName, HttpWebRequest request) { request = (HttpWebRequest)WebRequest.Create(uri); HttpWebResponse response = (HttpWebResponse)request.GetResponse(); Stream resStream = response.GetResponseStream(); StreamReader streamReader = new StreamReader(resStream, System.Text.Encoding.Default); string content = streamReader.ReadToEnd(); Business.IO.WriteFile(FileName, content); return true; }
把HttpWebRequest的Timeout在方法外面赋值,就不用每下载一个文件赋值一次。这样改后就搞定超时问题。下载大文件就采用异步处理,源码如下:
private bool DownloadBigFile(string uri, string FileName) { //创建文件 if (!File.Exists(FileName)) { FileStream s = File.Create(FileName); s.Close(); } else { FileStream s = new FileStream(FileName, FileMode.Create, FileAccess.Write); s.Close(); } //请求URL WebRequest request = WebRequest.Create(uri); request.Timeout = TIME_OUT; RequestState state = new RequestState(FileName); state.Req = request; //异步获取 request.BeginGetResponse(new AsyncCallback(RespCallback), state); //线程等侍 reset.WaitOne(); return true; } //返回流回调方法 private void RespCallback(IAsyncResult ar) { RequestState state = (RequestState)ar.AsyncState; WebRequest request = state.Req; //获取返回流 WebResponse response = request.EndGetResponse(ar); state.Res = response; Stream stream = response.GetResponseStream(); IAsyncResult isar = stream.BeginRead(state.Buffer, 0, BUFFER_SIZE, new AsyncCallback(ReadCallack), state); } //读取字符串回调方法 private void ReadCallack(IAsyncResult ar) { RequestState state = (RequestState)ar.AsyncState; WebResponse response = state.Res; Stream stream = response.GetResponseStream(); int i = stream.EndRead(ar); if (i > 0) { state.WriteByte(i); stream.BeginRead(state.Buffer, 0, BUFFER_SIZE, new AsyncCallback(ReadCallack), state); } else { //读取完成 stream.Close(); //关闭文件输出流 state.Close(); reset.Set(); } return; } //异步状态私有类 private class RequestState { private static Stream stream; public byte[] Buffer; public WebRequest Req; public WebResponse Res; public RequestState(string FileName) { Buffer = new byte[BUFFER_SIZE]; stream = new FileStream(FileName, FileMode.Append, FileAccess.Write); } public void WriteByte(int i) { stream.Write(Buffer, 0, i); stream.Flush(); } public void Close() { stream.Close(); } }
0
相关文章