copy azure storage containers to another storage account - Powershell - AzCopy

Powershell is very handy when you need some quick way of moving your data around. Some time back i was thinking that, moving data between azure storage account is very rare task, and you may not have good reason to do it.

Some time you need to move data between storage account, which are in different azure subscription.

In any case there can be different way of doing same task. recently i was given a task to move azure storage account blobs from one storage account to another storage account manually :).

May be i am lazy , but moving data manually from one storage account to another is error prune task.

I was looking for a automated way , to do manual assignment.

And to achieve that i have written little powershell script , which did my job very easily.

Here is the script.

#Start Writing Log to target file
Start-Transcript -path D:\AzCopyLog.log -append

#Path to AzCopy.exe, look at https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/ for more help
$AzCopyPath='C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\AzCopy'

$AzCopySourceAccountName='<SOURCE-STORAGE-ACCOUNT>'
$AzCopySourceAccountKey='<SOURCE-STORAGE-KEY>'
$AzCopyDestAccountName='<DEST-STORAGE-ACCOUNT>'
$AzCopyDestAccountKey='<DEST-STORAGE-KEY>'

#Get Storage Account Context for listing all containers
$Ctx = New-AzureStorageContext -StorageAccountName $AzCopySourceAccountName 
-StorageAccountKey $AzCopySourceAccountKey

#List of containers on source storage account.
$list=Get-AzureStorageContainer -Context $Ctx | select name

#Iterate over all containers, and execute AzCopy command for each container.
foreach ( $item in $list ) { 

  #Container name  
  $container=$item.Name

  #Log container name
  ECHO 'Container Name:'`t$container

  #Execure AzCopy Command
  CMD /C $AzCopyPath /Source:https://$AzCopySourceAccountName.blob.core.windows.net/$container /Dest:https://$AzCopyDestAccountName.blob.core.windows.net/$container /SourceKey:$AzCopySourceAccountKey /DestKey:$AzCopyDestAccountKey /S /Y

 }

 ECHO '#'$list.Count 'containers, successfully copied.'

Stop-Transcript

Most of the script is self explanatory. I have used AzCopy.exe to achieve this task. we can write same code in C#, using windows azure storage client library. 

We can leverage more options of AzCopy command at
https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/

How to download file using REST API OR Asp.Net

Long time back when i was working on asp.net project, there was very easy way to download file. 

You can write your file download code in Button_Click it self, and and you are done!!
Based on your requirement , using appropriate response header and content type, we can easily achieve file download.

Now in modern era of web-development we moved ahead of full post back approach,to ajax based implementation. with the advent of angularJs and other similar framework, we almost lost concept of back-end file.
We have RESTful services as back-end, and single page as front-end. How file file download is changed from asp.net to RESTful service?, is it really changed ?

In Asp.Net , we need to Simpy create back-end file, and handle click event of button.in simple case, below peace of code should be enough to achieve file download.

 protected void btnDownload_Click(object sender, EventArgs e)
 {
 Response.ContentType = "application/octet-stream";
 Response.AppendHeader("Content-Disposition", "attachment; filename=Sample.csv");
 Response.TransmitFile(Server.MapPath("~/App_Data/Sample.csv"));
 Response.End();
 }

OR

   protected void btnDownload_Click(object sender, EventArgs e)

  {
 Response.ContentType = "application/octet-stream";
 Response.AppendHeader("Content-Disposition", "attachment; filename=Sample.csv");
 StreamWriter writer = new StreamWriter(Response.OutputStream);
 FileStream fs = new FileStream(Server.MapPath("~/App_Data/Sample.csv"), FileMode.Open, FileAccess.Read);
 fs.CopyTo(Response.OutputStream);
 fs.Flush();
 Response.End();
  }

Now moving to web API file download, we just need a HttpGet endpoint, which takes file name as argument, and returns file as response.
In simple case, below is the method to achieve file download from asp.net WEB API.

   <a href="/api/file/DownloadFile?fileName=sample.csv">Download!!</a>

      [HttpGet]

        public IHttpActionResult DownloadFile(string fileName)
        {
            IHttpActionResult actionResult = null;
            var fileResponse = new HttpResponseMessage(HttpStatusCode.OK);

            FileStream fs = new FileStream(HostingEnvironment.MapPath("~/App_Data/Sample.csv"), FileMode.Open, FileAccess.Read);
            fileResponse.Content = new StreamContent(fs);
            fileResponse.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
            fileResponse.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
            fileResponse.Content.Headers.ContentDisposition.FileName = Path.GetFileName(fileName);
            actionResult = ResponseMessage(fileResponse);
            return actionResult;
        }

If we see both approach closely , there is nothing changed much, mostly syntax is changed. 

Conclusion is whatever changes in syntax, web technology is still web technology, let's focus on technology instead of syntax.
Hope everybody in on same page on this :) , regardless, hope this post will help in downloading file.